Different sources, one measurement language

The same heart rate can arrive from two sources with different units, frequencies and time formats. In EnaResearch each metric is defined in a canonical catalog: its unit, sampling type (continuous, interval or spot), valid value range and which sources can provide it from which platform data. The catalog holds about 197 device metrics today; manually entered measurements (such as pain scales and lab values) have been added as a separate source type.

The catalog is versioned. When a definition or transformation changes, the old one is not overwritten; a new version is added, and you know which transformation version produced each measurement. The source’s own unit, measurement time and device or app identity are preserved.

The mobile app sends data in batches; validation, normalization, de-duplication and durable storage are the server’s job. Batches pass schema and size checks. Records outside the catalog range or outside the contract are rejected at ingest with a reason, which is reported in the batch status.

A resent batch is not written again. A measurement key consists of person, metric, start time and source, so the same measurement arriving twice stays a single record. Cumulative metrics such as steps and active energy are collected as 5-minute buckets through the platform’s de-duplicating aggregate query, so overlapping phone and watch samples are not counted twice.

Time is resolved to UTC, and each measurement is linked to the person’s participation window and the study’s permitted metrics and sources. Data for the period after a study ends is rejected; late data for the period before is accepted for the window set on the study.

An invalid value (for example outside the catalog range) never enters the data. A value that is accepted but looks suspicious is flagged. A flag never changes or deletes the value; raw and standardized data stay as they are, and flags are computed at read time with the same rules. The rules are versioned and their thresholds are explained in the interface.

Completeness is the share of days in the selected period that have data, computed per participant and metric. The study’s Data quality page shows, for each metric, the number of participants with data, the median completeness and flag counts, plus a summary per participant; the period can be 30 days, 90 days or 1 year. Only the study team sees the page.

The participant panel also shows completeness and flagged days per metric; measurement values are not shown there. Completeness uses the same count as the data-coverage strip in the participant panel.

For each source, the last batch, the last processed batch, total rejections and the last error are kept; for each source and metric, the last measurement time, last receipt time and record count. Researchers see only the study’s permitted metrics in the Data transfer window on the participant row. The participant web area shows connected sources, last sync and a daily summary of the last 7 days; individual measurements are not shown.

In measurement datasets, raw rows carry their flags and aggregated rows carry the number of flagged samples. Every export includes quality.csv with days, days with data, completeness, samples and flag counts per participant and metric; the manifest records the rules applied and the totals. Quality information is computed from the data as of the cut-off; the same definition and cut-off give the same content hash.

Raw events are kept for 90 days and are never changed while kept. Standardized data is a separate record derived from them and is kept according to the study’s retention policy.

Data quality checks support the consistency of research data. They do not claim clinical accuracy or medical validity of measurements; whether data is fit for a study is assessed by the research team under the protocol.