-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 09fa64f
Showing
17 changed files
with
1,293 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
name: Deploy Azure Infrastructure with Bicep | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
workflow_dispatch: | ||
|
||
jobs: | ||
deploy-infrastructure: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set up Azure CLI | ||
uses: azure/cli@v2.1.0 | ||
with: | ||
azcliversion: 'latest' | ||
inlineScript: | | ||
# Log in to Azure | ||
az login --service-principal -u ${{ secrets.AZURE_CLIENT_ID }} -p ${{ secrets.AZURE_CLIENT_SECRET }} --tenant ${{ secrets.AZURE_TENANT_ID }} | ||
# Deploy Bicep Template | ||
az deployment group create --resource-group <your-resource-group> --template-file infrastructure/azure-resources.bicep |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
.vs/CQRS-Pattern-Azure/config/applicationhost.config | ||
.vs/ | ||
app-service-api/bin/Debug/net8.0/app-service-api.pdb | ||
app-service-api/bin/ | ||
app-service-api/obj/Debug/net8.0/app-service-api.csproj.CoreCompileInputs.cache | ||
app-service-api/obj/app-service-api.csproj.nuget.g.targets | ||
app-service-api/obj/Debug/net8.0/.NETCoreApp,Version=v8.0.AssemblyAttributes.cs | ||
app-service-api/obj/ | ||
update-processor/.vscode/ | ||
query-handler/.vscode/extensions.json | ||
command-handler/.vscode/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
MIT License | ||
|
||
Copyright (c) 2024 Igor Iric | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,116 @@ | ||
# Index Table Pattern on Azure | ||
|
||
This repository demonstrates the **Index Table Pattern** on Azure, which uses Azure Table Storage to store data and Azure AI Search for efficient indexing and searching of that data. The architecture is designed for scenarios where quick and scalable search capabilities are needed over structured data. | ||
|
||
## ποΈ Architectural Overview | ||
|
||
The **Index Table Pattern** architecture consists of the following components: | ||
|
||
1. **Azure Function**: Handles data updates and triggers indexing in Azure AI Search. | ||
2. **Azure Table Storage**: Stores the data in a scalable, low-cost NoSQL store. | ||
3. **Azure AI Search**: Indexes the data stored in Azure Table Storage for fast search capabilities. | ||
4. **Application Insights**: Monitors and collects telemetry data from the Azure Function. | ||
|
||
### π Architectural Diagram | ||
|
||
```mermaid | ||
graph TD | ||
Client["Client"] -->|Data Update| AzureFunction["Azure Function"] | ||
AzureFunction -->|Updates| AzureTableStorage["Azure Table Storage"] | ||
AzureFunction -->|Indexes| AzureAISearch["Azure AI Search"] | ||
Client -->|Search Request| AzureAISearch | ||
subgraph Monitoring | ||
AppInsights["Application Insights"] | ||
end | ||
AzureFunction -->|Logs & Telemetry| AppInsights | ||
``` | ||
|
||
## π Repository Structure | ||
|
||
``` | ||
/index-table-pattern | ||
β | ||
βββ README.md # Root README with architecture overview and getting started | ||
βββ LICENSE # MIT License | ||
β | ||
βββ infrastructure | ||
β βββ README.md # README for Infrastructure deployment | ||
β βββ azure-resources.bicep # Bicep template for all Azure resources | ||
β βββ .github/workflows/deploy-bicep.yml # GitHub Action to deploy Azure resources | ||
β | ||
βββ azure-functions | ||
βββ README.md # README for Index Table Function | ||
βββ IndexTableFunction.csproj # C# project file for Index Table Function | ||
βββ IndexTableFunction.cs # Main code for Index Table Function | ||
βββ .github/workflows/deploy-index-function.yml # GitHub Action to deploy Index Table Function | ||
``` | ||
|
||
## π Getting Started | ||
|
||
### Step 1: Deploy the Infrastructure | ||
|
||
1. **Navigate to the `infrastructure` Folder**: | ||
- Go to the **`infrastructure`** folder and follow the instructions in the [Infrastructure README](infrastructure/README.md) to deploy the required Azure resources using the Bicep template and GitHub Actions. | ||
|
||
2. **Run the GitHub Action**: | ||
- The GitHub Actions workflow **`deploy-bicep.yml`** will automatically deploy the Azure resources defined in the Bicep template. | ||
|
||
3. **Verify Deployment**: | ||
- After the deployment completes, verify that the Azure resources (Storage Account, Function App, AI Search, Application Insights) are properly created in your Azure subscription. | ||
|
||
### Step 2: Deploy the Azure Function | ||
|
||
1. **Navigate to the `azure-functions` Folder**: | ||
- Go to the **`azure-functions`** folder and follow the instructions in the [Function README](azure-functions/README.md) to deploy the Azure Function using GitHub Actions. | ||
|
||
2. **Run the GitHub Action**: | ||
- The GitHub Actions workflow **`deploy-index-function.yml`** will automatically build and deploy the Azure Function. | ||
|
||
### Step 3: Test the Azure Function | ||
|
||
1. **Use Postman or Any HTTP Client**: | ||
- Follow the testing instructions in the [Function README](azure-functions/README.md) to send HTTP POST requests to the Azure Function to update data and verify that it is correctly stored in Azure Table Storage and indexed in Azure AI Search. | ||
|
||
## π‘ How It Works | ||
|
||
1. **Data Update**: | ||
- The **Azure Function** receives HTTP POST requests with JSON data, updates the data in **Azure Table Storage**, and then triggers indexing in **Azure AI Search**. | ||
|
||
2. **Search Capabilities**: | ||
- **Azure AI Search** provides search capabilities over the data stored in Azure Table Storage, enabling quick and efficient searches. | ||
|
||
3. **Monitoring**: | ||
- **Application Insights** collects telemetry data from the Azure Function, allowing you to monitor its performance, track errors, and gain insights into usage patterns. | ||
|
||
## π Key Points to Remember | ||
|
||
- **Environment Variables**: Ensure all required environment variables are properly set in your Azure Function App: | ||
- **`AzureWebJobsStorage`**: Connection string for the Azure Storage Account. | ||
- **`AzureAISearchEndpoint`**: Endpoint URL for your Azure AI Search service. | ||
- **`AzureAISearchApiKey`**: API key for your Azure AI Search service. | ||
- **Monitor with Application Insights**: Check Application Insights to monitor the function's execution and gather telemetry data. | ||
|
||
## π Monitoring and Logging with Application Insights | ||
|
||
This repository uses **Azure Application Insights** to monitor and collect telemetry data from the Azure Function. Application Insights helps to: | ||
|
||
- Track request rates, response times, and failure rates. | ||
- Monitor the performance of the Azure Function. | ||
- Diagnose failures and exceptions. | ||
- Gain insights into the usage patterns and overall health of the system. | ||
|
||
### How to View Application Insights Data | ||
|
||
1. Go to the **Azure Portal**. | ||
2. Navigate to **Application Insights** and select the **`indexTableAppInsights`** resource. | ||
3. Use the available tools to explore logs, requests, failures, dependencies, and custom metrics. | ||
|
||
## π License | ||
|
||
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. | ||
|
||
## π Contributing | ||
|
||
Contributions are welcome! Please open an issue or submit a pull request for any improvements or suggestions. |
33 changes: 33 additions & 0 deletions
33
azure-functions/.github/workflows/deploy-index-function.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
name: Deploy Index Table Function | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
workflow_dispatch: | ||
|
||
jobs: | ||
build-and-deploy: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set up .NET | ||
uses: actions/setup-dotnet@v2 | ||
with: | ||
dotnet-version: '8.0.x' | ||
|
||
- name: Build the project | ||
run: dotnet build azure-functions/index-table-function/IndexTableFunction.csproj --configuration Release | ||
|
||
- name: Publish the project | ||
run: dotnet publish azure-functions/index-table-function/IndexTableFunction.csproj --configuration Release --output ./output | ||
|
||
- name: Deploy to Azure Function | ||
uses: Azure/functions-action@v1 | ||
with: | ||
app-name: 'indexTableFunctionApp' | ||
package: './output' | ||
publish-profile: ${{ secrets.AZURE_FUNCTIONAPP_PUBLISH_PROFILE }} |
Oops, something went wrong.