Core-Web-Hub

How to Run the Core Web Project

This document provides detailed instructions on how to set up and run the Core Web project, including information about all services, their ports, and how to start them.

Project Overview

The Core Web project is a comprehensive web platform built with Rust that includes multiple services and components:

1. Main Web Server - The primary application server
2. Jobs Worker - Background job processing service
3. WebSocket Gateway - Real-time communication gateway
4. CLI Tool - Command-line interface for managing the application
5. Database Services - MySQL, Redis, MongoDB, ClickHouse
6. Observability Services - OpenTelemetry collector for monitoring

Prerequisites

Before running the project, ensure you have the following installed:

• Rust (latest stable version)
• Docker and Docker Compose
• Git

Services and Ports

Web-Based Services

Service	Port	Description
Main Web Server (HTTP/1.1, HTTP/2)	8080	Primary application server with REST API endpoints
Main Web Server (HTTP/3)	4433	HTTP/3 server (when enabled with feature flag)
Metrics Server	8081	Prometheus metrics endpoint
MySQL Database	3306	Primary relational database
Redis	6379	In-memory data structure store
MongoDB	27017	Document-based database
ClickHouse	8123 (HTTP), 9000 (TCP)	Analytical database for analytics
OpenTelemetry Collector	4317 (gRPC), 4318 (HTTP)	Observability data collection

Terminal-Based Services

Service	Command	Description
Core CLI	cargo run -p core-cli -- [command]	Command-line interface for managing the application
Database Migrations	./scripts/migrate.sh	Apply database schema changes
Code Generation	./scripts/generate-code.sh	Generate code from specifications
Development Environment	./scripts/dev-up.sh	Start all development services
Environment Shutdown	./scripts/dev-down.sh	Stop all development services

Running the Project

1. Start Database and Supporting Services

First, start all the required database services and observability tools:

./scripts/dev-up.sh

This script will:

• Create necessary data directories
• Start MySQL, Redis, MongoDB, ClickHouse, and OpenTelemetry collector using Docker Compose
• Display the ports on which each service is running

2. Run the Main Web Server

To start the main web server:

cd apps/core-web
cargo run

The server will start on port 8080 by default. You can access the API at http://localhost:8080.

3. Run the Jobs Worker

To start the background jobs worker:

cd apps/jobs-worker
cargo run

4. Run the WebSocket Gateway

To start the WebSocket gateway:

cd apps/ws-gateway
cargo run

5. Using the CLI Tool

The CLI tool provides various commands for managing the application:

# Navigate to the CLI crate
cd crates/core-cli

# Run CLI commands
cargo run -- --help                    # Show help
cargo run -- start                     # Start the server
cargo run -- stop                      # Stop the server
cargo run -- status                    # Check server status
cargo run -- health                    # Run health checks
cargo run -- logs                      # View logs
cargo run -- config show               # Show configuration
cargo run -- db migrate                # Run database migrations

API Endpoints

Health and Monitoring

• GET /healthz - Basic health check
• GET /readyz - Readiness probe
• GET /health - Advanced health check with component status
• GET /metrics - Prometheus metrics (available on port 8081)

User Management

• GET /api/v1/users - List all users
• POST /api/v1/users - Create a new user
• GET /api/v1/users/{id} - Get a specific user
• PUT /api/v1/users/{id} - Update a specific user
• DELETE /api/v1/users/{id} - Delete a specific user

Authentication

• POST /api/v1/auth/login - User login endpoint

SDK Generation

• POST /sdk/generate - Generate SDKs for various platforms
• POST /docs/generate - Generate API documentation

Feature Flags

The project supports several feature flags that can be enabled during compilation:

• http3 - Enable HTTP/3 support
• pprof - Enable CPU profiling
• mysql - Enable MySQL support
• redis - Enable Redis support
• mongo - Enable MongoDB support
• clickhouse - Enable ClickHouse support
• openapi - Enable OpenAPI support
• graphql - Enable GraphQL support
• grpc - Enable gRPC support
• otel - Enable OpenTelemetry support

To run with specific features:

cd apps/core-web
cargo run --features "http3,pprof"

Configuration

The application can be configured using TOML files located in the configs/ directory:

• development.toml - Development environment configuration
• staging.toml - Staging environment configuration
• production.toml - Production environment configuration

Environment variables can also be used to override configuration values:

• DATABASE_URL - Database connection string
• JWT_SECRET - Secret key for JWT token generation
• SERVER_PORT - Port for the main web server

Testing

To run the test suite:

# Run all tests
cargo test

# Run tests for a specific crate
cd crates/core-analytics
cargo test

Building for Production

To build optimized binaries for production:

# Build all workspace members
cargo build --release

# Build specific binary
cd apps/core-web
cargo build --release

The compiled binaries will be located in the target/release/ directory.

Troubleshooting

Common Issues

1. Port conflicts: If you see port conflicts, make sure no other services are running on the required ports or modify the configuration files to use different ports.

2. Docker issues: Ensure Docker is running and you have sufficient permissions to run Docker commands.

3. Database connection errors: Verify that the database services are running and the connection strings in the configuration files are correct.

Logs

Application logs are output to the console by default. For more detailed logging, you can configure the logging level using environment variables:

RUST_LOG=debug cargo run

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details on how to get started.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a pull request

Please note that our main branch is protected with required status checks and code reviews. All pull requests must pass our CI pipeline and receive approval before merging.

🛡️ Repository Security

This repository implements several security measures:

• Branch Protection: Main branch requires status checks and code reviews
• Automated Security Scanning: CodeQL and dependency scanning
• Security Audits: Regular vulnerability scanning
• Secure Development Practices: Following Rust security guidelines

For detailed information about our security practices, see SECURITY.md.

Additional Resources

• API Documentation
• Architecture Overview
• Crate Stack
• Feature Flag Matrix
• Reference Endpoints

This site is open source. Improve this page.