receive new data;

store the new data in a first data structure in the staging datastore, wherein one or more fields of the first data structure have a predefined relationship with one or more fields of a second data structure in the operational datastore;

receive a first query configured to query and retrieve data from the first data structure, the first query comprising at least one field in the first data structure on which the relationship between the data structures is defined;

periodically migrate data from the first data structure to the second data structure according to the predefined relationship, wherein the migrating occurs in response to a predefined condition being met; and

transform non-migrated data into a format consistent with the format of the second data structure; and

receive a user defined query configured to query and return data from the operational datastore;

in response to receiving the user defined query, execute the first query to retrieve, from the first data structure, any non-migrated data relevant to the user defined query; and

execute the user defined query against a data set including the second data structure and the transformed data.