[FEA] Update missing instance types in pricing info file for Databricks platforms

[cindyyuanjiang]

Is your feature request related to a problem? Please describe.
In the current user tools implementation, we store instance type pricing information in static files premium-databricks-azure-catalog.json and databricks-premium-catalog.json. We need to update these json files to reflect the newly added instance types on the official Databricks pricing website.

Ref:
https://www.databricks.com/en-website-assets/data/pricing/AWS.json
https://azure.microsoft.com/en-us/pricing/details/databricks/

Describe the solution you'd like
We should update the files to include all new instance types. In addition, we should have an automated process to retrieve and update the pricing information to avoid this issue in the future.

Tasks

• Skip cost savings if pricing info is missing for Databricks qual tool #1051
• Refactor DB AWS qual tool to cache and process pricing info from DB website #1139
• DB Azure pricing info is outdated and should be regularly updated

[cindyyuanjiang]

Recently found: https://prices.azure.com/api/retail/prices, this could be useful for databricks azure pricing. The website only contains price for azure, we need to add DBU costs ourselves.

[cindyyuanjiang]

Closing this as tools is removing support for cost savings. https://prices.azure.com/api/retail/prices could be a useful resource for Azure pricing if we need to revisit this in the future.