Preparing for time series based on a datetime or numeric key field with constant separation
Created
Thu, 8 Oct 2026 15:01:23 +0000
Published
Thu, 8 Oct 2026 17:26:19 +0000
Preparing time series data with flaws. The kind of problems addressed in the script are:
Unsorted rows: Data may not be in an ascending sequence
Gaps between rows: Some of the rows may be missing, according to the values found in the epoch or datetime field used as key to the sequence and the user-given lapse to be used
Multiple rows: There might be more than one row per key-field value. If that happens, the script will keep the first row.
The process sorts the rows according to the given key-field, fills the gaps as defined by it and the expected lapse to be found between rows and removes multiplicities. The result will be a new timeseries-normalized dataset that can be used to build a time series from it.
Input Parameters
dataset-id: ID of the original dataset to be fixed
- key-field: Name of the field that contains the key to the time series sequence. The field is expected to be either a numeric (possibly epoch) field or a datetime field. If the latter is used, the dt-field-format field is used to define the original field parsing.
dt-field-format: Format used to parse the key-field information (only used if that's a datetime field).
lapse: Uniform interval expected between the consecutive key-field values.
Output
Time series normalized dataset: It will contain a sorted and uniformely sequence of rows. If the key-field used in the original dataset is a datetime field, a new field named after it with an _epoch suffix will be added to it containing the epoch equivalence of the original timestamps. It also adds a new day_number that stores the consecutive number of days.
Preparing time series data with flaws. The kind of problems addressed in the script are:
Unsorted rows: Data may not be in an ascending sequence
Gaps between rows: Some of the rows may be missing, according to the values found in the epoch or datetime field used as key to the sequence and the user-given lapse to be used
Multiple rows: There might be more than one row per key-field value. If that happens, the script will keep the first row.
The process sorts the rows according to the given key-field, fills the gaps as defined by it and the expected lapse to be found between rows and removes multiplicities. The result will be a new timeseries-normalized dataset that can be used to build a time series from it.
Input Parameters
dataset-id: ID of the original dataset to be fixed
- key-field: Name of the field that contains the key to the time series sequence. The field is expected to be either a numeric (possibly epoch) field or a datetime field. If the latter is used, the dt-field-format field is used to define the original field parsing.
dt-field-format: Format used to parse the key-field information (only used if that's a datetime field).
lapse: Uniform interval expected between the consecutive key-field values.
Output
Time series normalized dataset: It will contain a sorted and uniformely sequence of rows. If the key-field used in the original dataset is a datetime field, a new field named after it with an _epoch suffix will be added to it containing the epoch equivalence of the original timestamps. It also adds a new day_number that stores the consecutive number of days.
Preparing time series data with flaws. The kind of problems addressed in the script are:
Unsorted rows: Data may not be in an ascending sequence
Gaps between rows: Some of the rows may be missing, according to the values found in the epoch or datetime field used as key to the sequence and the user-given lapse to be used
Multiple rows: There might be more than one row per key-field value. If that happens, the script will keep the first row.
The process sorts the rows according to the given key-field, fills the gaps as defined by it and the expected lapse to be found between rows and removes multiplicities. The result will be a new timeseries-normalized dataset that can be used to build a time series from it.
Input Parameters
dataset-id: ID of the original dataset to be fixed
- key-field: Name of the field that contains the key to the time series sequence. The field is expected to be either a numeric (possibly epoch) field or a datetime field. If the latter is used, the dt-field-format field is used to define the original field parsing.
dt-field-format: Format used to parse the key-field information (only used if that's a datetime field).
lapse: Uniform interval expected between the consecutive key-field values.
Output
Time series normalized dataset: It will contain a sorted and uniformely sequence of rows. If the key-field used in the original dataset is a datetime field, a new field named after it with an _epoch suffix will be added to it containing the epoch equivalence of the original timestamps. It also adds a new day_number that stores the consecutive number of days.
