Pigeon

Pigeon is a monolotic clean architecture .Net dual-purpose project designed to help website owners monitor and manage the status of their web pages and aslo perform data mining to extract insights from news publisher websites.

Features

Part 1: Web Crawler and Monitoring

• Crawls websites to extract mentioned links on a page. Link Crawler.cs
• Background job fetches and logs page response time and status code.
• Ticketing system for admins to assign crashed pages to team members and track resolutions (Admin interface).
• Login and user management with Asp .Net Identity

Part 2: Data Mining

• Extracts tags from news websites.
• the Apriori algorithm implementation to find relationships between tags. check this page DataMining.cs

Installation

1. Clone the repository:

   git clone https://github.com/abowfzl/Pigeon.git

2. Navigate to the project directory:

   cd Pigeon

3. Install dependencies:

   dotnet restore

Usage

1. Build and run the project:

   dotnet build
   dotnet run

2. The application will automatically start the web crawler, logging, and data mining processes.

Contributing

Contributions are welcome! Please follow these steps to contribute:

1. Fork the repository.
2. Create a new branch:

   git checkout -b feature/YourFeature

3. Make your changes.
4. Commit your changes:

   git commit -m 'Add some feature'

5. Push to the branch:

   git push origin feature/YourFeature

6. Open a pull request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contact

For any questions or inquiries, please contact Abolfazl Moslemian at [moslemianabolfazl@gmail.com].