PostgreSQL Inventory¶

Rationale¶

Starting with 9.0, and back ported to 8.0.15 we have added a PostgreSQL tool to make sure no data is lost after a migration.

Usage¶

Script can be run via: python3 ./support/postgres/model_monitoring_inventory.py inside DR environment

Optional parameters are: * -connection_string, that can be used to pass db connection parameters in the form of postgresql://scott:tiger@localhost/test (default value will be pulled from the environment) * -output_file to change where inventory will be written (default is inventory.json)

Output of the script is a json file in a form of

{
    "users": [
        "metering_api",
        "metering_worker",
        "modmon",
        "postgres"
    ],
    "tables": [
        {
            "schema": "public",
            "name": "modmon_batch",
            "row_count": 0,
            "indexes": [
                  {
                    "name": "modmon_batch_ts_max",
                    "unique": false,
                    "column_names": ["ts_max"]
                  }
            ]
        }
    ]
}

Inventory should be produced before and after running data migration. ./support/postgres/model_monitoring_compare_inventory.py <path_to_pre_migration_inventroy> <path_to_post_migration_inventory> can be used to compare two inventories and will output the diff between them. Example output

DIFFERENT TABLES DETECTED
tables before not after:
tables after not before: mpcr_2
TABLE modmon_prediction_feature_stats is different before and after
TABLE modmon_training_classification_stats is different before and after

Scripts can also be used on prior 8.0 versions (8.0.x), for that contents of ./support/postgres/ directory need to be mounted in the DR environment