A method for managing data in an operational datastore and a staging datastore comprising: configuring the staging datastore, comprising: (a) defining a relationship between one or more fields of a first data structure in the staging datastore and one or more fields of a second data structure in the operational datastore, wherein the first data structure is a data source for the second data structure in the operational datastore; and (b) defining a first query comprising at least one field in the first data structure on which the relationship between the data structures is defined, the first query being configured to query and retrieve data from the first data structure; periodically migrating data from the first data structure to the second data structure according to the defined relationship, wherein the migrating occurs in response to a predefined condition being met; receiving a user defined query configured to query and return data from the operational datastore; in response to receiving the user defined query, executing the first query to retrieve, from the first data structure, any non-migrated data relevant to the user defined query; in the event any non-migrated data relevant to the user defined query are returned, transforming the returned non-migrated data into a format consistent with the format of the second data structure; executing the user defined query against a data set including the second data structure and the transformed data; and returning results for the executed user defined query.
