Data from the "Rewarding the Original" trials
"Rewarding the Original" is a system/experimental protocol for exploring the spaces of motions usable for communication with a given user/input-device combination. It is described in:
Rewarding the Original: Explorations in Joint User-Sensor Systems, J. Williamson and R. Murray-Smith ACM SIGCHI'12 , 2012 PDF.
Abstract: This paper presents a systematic and general technique for
establishing a set of motions suitable for use with sensor
systems, by drawing performable and measurable motions
directly from users. It uses reinforcement which rewards
originality to induce users to explore the space of motions
they can perform. A decomposition of movements into motion
primitives is constructed, among which a meaningful
originality metric can be defined. Because the originality
measure is defined in terms of the sensed input, the resulting
space contains only movements which can both be performed
and sensed. We show how this can be used to evaluate
the relative performance of different joint user-sensor
systems, providing objective analyses of gesture lexicons
with regard to the technical limitations of sensors and humans.
In particular, we show how the space of motions
varies across the arm for a body-mounted inertial sensor.
Video
A video is available: motionexplorer.mp4 (high quality, MP4) or it can be seen on YouTube:Data
The data captured during the experiment is made available for other researchers to use as they wish. Please cite the above paper when making use of this data set. Unfortunately the recorded video data cannot be made available as it compromises subject anonymity.Download
This includes the raw sensor stream from the inertial sensor, the processed motion vectors and the status logs (which include logs of the feedback levels and the packet loss levels).
There were 20 participants in the trial; each has an anonymised two-letter tag. There is 18 minutes of data for each participant; 2 conditions with 3 sub-trials of 3 minute duration. The description of the data files is given in detail below.
Experimental procedure
The experimental procedure from the paper is reproduced below (see the paper for full technical details):Experiment Structure
The inertial sensors are sampled synchronously at 32Hz; the sensor streams are filtered on the hardware to bandlimit them before decimating from an original sampling rate of 1024Hz. A 32Hz rate is sufficient to capture the details of most limb movement. At a 0.65s window length one motion primitive is therefore extracted from 21 samples of data (the window time is adjusted slightly to be exactly 21 samples in length). A bank of Savitsky-Golay filters are applied to the signal, each designed to estimate one smoothed derivative efficiently. These coefficients are concatenated into a single vector M with q = 9 x 4 = 36 dimensions, which represents the motion in sensor space.
Hardware
The novelty detector captures data from a SHAKE SK7 inertial sensor, which provides 3-axis accelerometer, gyroscope and magnetometer outputs. The accelerometer, gyroscope and magnetometer measure in the range -6g – 6g, -900 – 900 degrees/second and -0.2mT – 0.2mT respectively, all with 12 bit resolution. The sensor is attached with an elastic band.Tasks
The exploration was carried out in two conditions: condition A, with the sensor attached to the dominant hand; and condition B, with the sensor attached to the arm just above the elbow.Procedure
The experiment was conducted in a laboratory. There were N = 20 participants, 16 male, 4 female, recruited from a pool of computer science students, with an age range of 18–40. All participants gave informed consent. The SK7 sensor pack was affixed with an elastic strap to maintain a firm connection.All participants completed both conditions in counterbalanced order. Each condition was conducted as a set of three subtrials, lasting three minutes, with a break between each. Each subtrial continued from the point the previous one left off, for a total of nine minutes of exploration in each condition. The conditions were broken into subtrials to combat fatigue from long performances. The implementation uses k = 5 nearest neighbours and a value of delta = 1.50 for the novelty cutoff.
Trial format
Each "sub-trial" creates a directory containing the data for that session. These directories are named XX_[hand|elbow]_n, where XX is the identifier for the participant, hand or elbow specifies where the sensor was mounted, and n can be 0, 1, or 2 in sequence.Within each directory, there are six log files (all plain ASCII text files):
baselog.txt Records basic attributes about the trial (start/end times, etc.)
Text file, in key: value format, one item per line
eventlog.txt Records overall "score" and feedback state (level of feedback output)
- start_time: "yyyy-mm-dd hh:mm:ss.dddddd" Start time of the trial
- window: nn Sliding window length in samples (will be 21 in all examples)
- order: n Polynomial order (will be 4 for all examples)
- sample_rate: nn.nnnnn Sample rate, in Hz (32 for all examples)
- cutoff: 16.000000 Cutoff for the lowpass filter (disabled if=Nyquist)
- continutation: XX_[hand|elbow]_n The directory this log continues from, if there is one.
- end_seconds: nnn Duration of the trial in seconds (will be 180 for all examples)
- end_time: "yyyy-mm-dd hh:mm:ss.dddddd" End time of the trial
Text file, one row per log time, space separated. Format:
real_time trial_time rscore feedback_level score locked added vectors dummy
rawdata.txt Raw sensor stream, 32Hz sampled from accelerometer, magnetometer and gyroscope.
real_time trial_time rscore feedback_level score locked added vectors dummy
- real_time "yyyy-mm-dd hh:mm:ss.dddddd" Wall clock time, as a quoted string
- trial_time Time in seconds since start of trial
- rscore Score of last vector (bigger = more original)
- feedback_level Level of feedback, as a floating point number
- score Smoothed score level
- lock 0 normally, 1 if the system is "locked" due to sensor saturation
- added 1 if the vector was added to the repertoire
- vectors Total number of unique vectors so far
- dummy Always 0
Format:
time sensor_clock ax ay az mx my mz gx gy gz drops
vectors.data The processed motion vectors for each submovement. Each vector is a point in 36-dimensional space representing the movement.
time sensor_clock ax ay az mx my mz gx gy gz drops
- time time, in floating point seconds, since the start of the trial
- sensor_clock timestamp from the sensor. Integer, runs from 0-255. Should increment by 1 every sample (except wrapping 255->0) -- if not a packet has been dropped
- ax,ay,az Accelerometer values (in milliGs).
- mx,my,mz Magnetometer values (in milliGauss).
- gx,gy,gz Gyroscope values (in degrees/second * 10, so 3600 = one revolution per second).
- drops Current packet drop estimator (smoothed).
Space separated floating point values, 36 values per line. Each vector is in the order it was added to repertoire. The vector is ordered:
[ax dax d2ax d3ax ay day ... az ... mx dmx d2mx ... gx ...]
where ax/gx/mx is the x-component of the accelerometer, gyroscope and magnetometer respectively, and dax means the first derivative of ax, d2ax means the second derivative etc.
vectors.cov The covariance matrix of the captured set of vectors (as a MATLAB-style text file).
[ax dax d2ax d3ax ay day ... az ... mx dmx d2mx ... gx ...]
where ax/gx/mx is the x-component of the accelerometer, gyroscope and magnetometer respectively, and dax means the first derivative of ax, d2ax means the second derivative etc.
A single text 36x36 matrix, which is the covariance of all the vectors for this subject.
config.py Internal parameters of the system, as a Python script file. Identical for all datasets and not very interesting for analysis!