hlaeja-old/hlaeja-development
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add redis and cleanup

Browse Source 
- add doc/global_settings.md
- docker_database.md
- update dependency.md
- extrack hlaeja from  development-compose.yml to hlaeja-compose.yml
- rename compose.yml to development-compose.yml
- extract text from readme.md
  - databases to doc/docker_database.md
  - hlaeja service to doc/docker_hlaeja.md
- move and rename structure.md to doc/dependency.md
- move and rename identity.md to doc/rsa_key.mk
- move and rename certificate.md to doc/keystore.md
This commit is contained in:
Mr. Swordsteel
2024-12-20 13:47:19 +01:00
parent 5dc1e91ce2
commit 614abcb272
12 changed files with 470 additions and 281 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
CERTIFICATE.md
View File

@@ -1,11 +0,0 @@
# Generate Keystore
### Generate Keystores
```shell
keytool -genkeypair -alias <service> -keyalg RSA -keysize 2048 -validity 3650 -dname "CN=<domain>" -keypass <password> -keystore ./keys/<keystore>.p12 -storetype PKCS12 -storepass <password>
```
### Export the public certificate
```shell
keytool -export -alias <service> -keystore ./keys/<keystore>.p12 -storepass <password> -file ./keys/<domain>.cer -rfc
```

123
README.md
View File

@@ -2,127 +2,58 @@
Services and networks, to shape and to steer, Containers in harmony, their roles made clear. Each config declared, each volume in place, Through Compose they unite, to streamline the space. Compose pathways, structured and strong, Linking apps to environments, where they belong. Bound by one purpose, to simplify all, Empowering development, answering the call. Services and networks, to shape and to steer, Containers in harmony, their roles made clear. Each config declared, each volume in place, Through Compose they unite, to streamline the space. Compose pathways, structured and strong, Linking apps to environments, where they belong. Bound by one purpose, to simplify all, Empowering development, answering the call.
## Version Catalog ## Setup
### Databases
Hlæja using different databases read [Database setup](./doc/docker_database.md)
### Hlæja Services
Hlæja consists of services read [service setup](./doc/docker_hlaeja.md)
## Repositories
Hlæja is a system build from Gradle plugins, libraries, and services, look at [dependencies](./doc/dependency.md) visualisation
### Version Catalog
Control all dependencies from a central location. GitHub [Hlæja Version Catalog](https://github.com/swordsteel/hlaeja-version-catalog) Control all dependencies from a central location. GitHub [Hlæja Version Catalog](https://github.com/swordsteel/hlaeja-version-catalog)
## Gradle Plugin ### Gradle Plugin
### Core Plugin #### Core Plugin
Plugin containing basic function ust in all repositories. GitHub [Hlæja Core Plugin](https://github.com/swordsteel/hlaeja-core-plugin) Plugin containing basic function ust in all repositories. GitHub [Hlæja Core Plugin](https://github.com/swordsteel/hlaeja-core-plugin)
### Common Plugin #### Common Plugin
Plugin containing gradle task and setting used by common, library, and service repositories. GitHub [Hlæja Common Plugin](https://github.com/swordsteel/hlaeja-common-plugin) Plugin containing gradle task and setting used by common, library, and service repositories. GitHub [Hlæja Common Plugin](https://github.com/swordsteel/hlaeja-common-plugin)
## Library ### Library
### Common Messages #### Common Messages
Library containing all internal messages for services. GitHub [Hlæja Common Messages](https://github.com/swordsteel/hlaeja-common-messages) Library containing all internal messages for services. GitHub [Hlæja Common Messages](https://github.com/swordsteel/hlaeja-common-messages)
## Services ### Services
### Device Data #### Device Data
Store measurement from electronic devices. GitHub [Hlæja Device Data](https://github.com/swordsteel/hlaeja-device-data) Store measurement from electronic devices. GitHub [Hlæja Device Data](https://github.com/swordsteel/hlaeja-device-data)
#### Environment #### Device Registry
```text
SPRING_PROFILES_ACTIVE: docker
INFLUXDB_TOKEN: influxdbToken==
```
### Device Registry
Store device information. GitHub [Hlæja Device Data](https://github.com/swordsteel/hlaeja-device-registry) Store device information. GitHub [Hlæja Device Data](https://github.com/swordsteel/hlaeja-device-registry)
#### Environment #### Device API
```text
SPRING_R2DBC_URL: r2dbc:postgresql://localhost:5432/device_registry
SPRING_R2DBC_USERBAME: services
SPRING_R2DBC_PASSWORD: password
JWT_PRIVATE_KEY: cert/private_key.pem
```
#### Volume
This will allow you to mount a local private key `identity_private_key.pem` into the container. Read `IDENTITY.md` for how to generate.
```text
volumes:
- ./cert/identity_private_key.pem:/app/resources/cert/private_key.pem
```
### Device API
Api for electronic devices. GitHub [Hlæja Device Data](https://github.com/swordsteel/hlaeja-device-api) Api for electronic devices. GitHub [Hlæja Device Data](https://github.com/swordsteel/hlaeja-device-api)
#### Environment #### Device Configuration
```text
SPRING_PROFILES_ACTIVE: docker
```
#### Volume
This will allow you to mount a local keystore `device_api_keystore.p12`, and local public key `identity_public_key.pem` into the container. Read `CERTIFICATE.md`, and `IDENTITY.md` for how to generate.
```text
volumes:
- ./keys/identity_public_key.pem:/app/resources/cert/public_key.pem
- ./keys/device_api_keystore.p12:/app/resources/cert/keystore.p12
```
### Device Configuration
Store configurations for electronic devices. GitHub [Hlæja Device Configuration](https://github.com/swordsteel/hlaeja-device-configuration) Store configurations for electronic devices. GitHub [Hlæja Device Configuration](https://github.com/swordsteel/hlaeja-device-configuration)
#### Environment #### Registry API
```text Store configurations for electronic devices. GitHub [Hlæja Registry API](https://github.com/swordsteel/hlaeja-registry-api)
SPRING_PROFILES_ACTIVE: docker
```
## Databases
### InfluxDB
InfluxDB is a high-performance time series database designed to handle large volumes of time-stamped data. It is commonly used for monitoring, analytics, and IoT applications, where data points are associated with timestamps (e.g., sensor readings, system metrics).
#### Environment
```text
DOCKER_INFLUXDB_INIT_MODE: setup
DOCKER_INFLUXDB_INIT_USERNAME: influx
DOCKER_INFLUXDB_INIT_PASSWORD: password
DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: influxdbToken==
DOCKER_INFLUXDB_INIT_ORG: hlaeja_ltd
DOCKER_INFLUXDB_INIT_BUCKET: device-data
```
### PostgreSQL
PostgreSQL is a powerful, open-source relational database management system (RDBMS). Known for its reliability and advanced features, it supports SQL for querying and managing data, along with extensive functionality for scalability and extensibility.
#### Environment
```text
POSTGRES_USER: postgres
POSTGRES_PASSWORD : password
```
### PostgreSQL
Apache Cassandra is a distributed NoSQL database designed for handling large amounts of data across many commodity servers with no single point of failure. It is optimized for high availability, scalability, and fault tolerance.
#### Environment
```text
CASSANDRA_USER: cassandra
CASSANDRA_PASSWORD: password
```

48
STRUCTURE.md
View File

@@ -1,48 +0,0 @@
# Hlæja structure
```mermaid
graph RL;
CP[Core Plugin]
subgraph SCP [Common Plugin]
PL[Plugin Library]
PLM[Plugin Library Manifest]
PLP[Plugin Library Publish]
PCo[Plugin Common]
PCoD[Plugin Common Detekt]
PCoK[Plugin Common Ktlint]
PS[Plugin Service]
PSC[Plugin Service Container]
PSIT[Plugin Service Integration Test]
PSPR[Plugin Service Process Resource]
PCe[Plugin Certificate]
end
PCo --> PL
PLM --> PL
PLP --> PL
CP --> PCo
PCoD --> PCo
PCoK --> PCo
PCo --> PS
PSC --> PS
PSIT --> PS
PSPR --> PS
CML[Common Messages Library]
PL --> CML
DRS[Device Registry Service]
CML --> DRS
PS --> DRS
PCe --> DRS
DDS[Device Data Service]
CML --> DDS
PS --> DDS
DCS[Device Configuration Service]
CML --> DCS
PS --> DCS
DAS[Device API Service]
CML --> DAS
PS --> DAS
PCe --> DAS
RAS[Registry API Service]
CML --> RAS
PS --> RAS
PCe --> RAS
```

126
compose.yml
View File

@@ -1,126 +0,0 @@
name: hlaeja-development
networks:
develop:
name: develop
volumes:
cassandra:
influx-config:
influx-data:
postgres:
services:
device-data:
image: hlaeja-device-data:0.1.1
container_name: DeviceData
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
INFLUXDB_TOKEN: influxdbToken==
networks:
- develop
ports:
- "9020:8080"
device-registry:
image: hlaeja-device-registry:0.2.0
container_name: DeviceRegistry
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
networks:
- develop
ports:
- "9010:8080"
# mount a local `identity_private_key.pem` into the container.
volumes:
- ./cert/identity_private_key.pem:/app/resources/cert/private_key.pem
device-api:
image: hlaeja-device-api:0.2.0
container_name: DeviceApi
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
networks:
- develop
ports:
- "9000:8443"
# mount a local `identity_public_key.pem` into the container.
volumes:
- ./cert/identity_public_key.pem:/app/resources/cert/public_key.pem
- ./cert/device_api_keystore.p12:/app/resources/cert/keystore.p12
device-configuration:
image: hlaeja-device-configuration:0.1.0
container_name: DeviceConfiguration
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
networks:
- develop
ports:
- "9030:8080"
registry-api:
image: hlaeja-registry-api:0.1.0-SNAPSHOT
container_name: RegistryApi
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
networks:
- develop
ports:
- "9040:8443"
# mount a local `identity_public_key.pem` into the container.
volumes:
- ./cert/registry_api_keystore.p12:/app/resources/cert/keystore.p12
influxdb:
image: influxdb:2.7.10-alpine
container_name: InfluxDB
restart: unless-stopped
environment:
DOCKER_INFLUXDB_INIT_MODE: setup
DOCKER_INFLUXDB_INIT_USERNAME: influx
DOCKER_INFLUXDB_INIT_PASSWORD: password
DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: influxdbToken==
DOCKER_INFLUXDB_INIT_ORG: hlaeja_ltd
DOCKER_INFLUXDB_INIT_BUCKET: device-data
networks:
- develop
ports:
- "8086:8086"
volumes:
- influx-data:/var/lib/influxdb2
- influx-config:/etc/influxdb2
postgres:
image: postgres:17.1-alpine
container_name: PostgreSQL
restart: unless-stopped
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD : password
ports:
- "5432:5432"
networks:
- develop
volumes:
- postgres:/var/lib/postgresql/data
cassandra:
image: cassandra:5.0.2
container_name: Cassandra
restart: unless-stopped
environment:
CASSANDRA_USER: cassandra
CASSANDRA_PASSWORD: password
networks:
- develop
ports:
- "9042:9042"
volumes:
- cassandra:/var/lib/cassandra

75
development-compose.yml Normal file
View File

@@ -0,0 +1,75 @@
name: development
networks:
develop:
name: develop
external: true
volumes:
cassandra:
influx-config:
influx-data:
postgres:
redis:
services:
influxdb:
image: influxdb:2.7.10-alpine
container_name: InfluxDB
restart: unless-stopped
environment:
DOCKER_INFLUXDB_INIT_MODE: setup
DOCKER_INFLUXDB_INIT_USERNAME: influx
DOCKER_INFLUXDB_INIT_PASSWORD: password
DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: influxdbToken==
DOCKER_INFLUXDB_INIT_ORG: hlaeja_ltd
DOCKER_INFLUXDB_INIT_BUCKET: device-data
networks:
- develop
ports:
- 8086:8086
volumes:
- influx-data:/var/lib/influxdb2
- influx-config:/etc/influxdb2
postgres:
image: postgres:17.1-alpine
container_name: PostgreSQL
restart: unless-stopped
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD : password
ports:
- 5432:5432
networks:
- develop
volumes:
- postgres:/var/lib/postgresql/data
cassandra:
image: cassandra:5.0.2
container_name: Cassandra
restart: unless-stopped
environment:
CASSANDRA_USER: cassandra
CASSANDRA_PASSWORD: password
networks:
- develop
ports:
- 9042:9042
volumes:
- cassandra:/var/lib/cassandra
redis:
image: redis:7.4.1-alpine
container_name: Redis
restart: unless-stopped
environment:
REDIS_PASSWORD: password
networks:
- develop
ports:
- 6379:6379
volumes:
- redis:/data

99
doc/dependency.md Normal file
View File

@@ -0,0 +1,99 @@
# Hlæja dependency
## Service dependency
```mermaid
graph TD
;
subgraph BE[Backend Services]
subgraph HDR[Hlæja Device Registry]
HDRS[Service] --> HDRD[(Postgres)]
end
subgraph HDD[Hlæja Device Data]
HDDS[Service] --> HDDD[(InfluxDB)]
end
subgraph HDC[Hlæja Device Configuration]
HDCS[Service] --> HDCD[(Cassandra)]
end
subgraph HAR[Hlæja Account Registry]
HARS[Service] --> HARD[(Postgres)]
end
end
subgraph HDA[Hlæja Device API]
HDAS[Service] --> HDAD[(Redis)]
end
subgraph HRA[Hlæja Registry API]
HRAS[Service]
end
subgraph HM[Hlæja Management]
HMS[Service]
end
HDA --> HDR
HDA --> HDC
HDA --> HDD
HRA --> HDR
HRA -.-> HAR
HM -.-> HDC
HM -.-> HDR
HM -.-> HAR
```
## Library and Gradle plugin dependency
```mermaid
graph RL
;
CP[Core Plugin]
subgraph SCP [Common Plugin]
PL[Plugin Library]
PLM[Plugin Library Manifest]
PLP[Plugin Library Publish]
PCo[Plugin Common]
PCoD[Plugin Common Detekt]
PCoK[Plugin Common Ktlint]
PS[Plugin Service]
PSC[Plugin Service Container]
PSIT[Plugin Service Integration Test]
PSPR[Plugin Service Process Resource]
PCe[Plugin Certificate]
end
PCo --> PL
PLM --> PL
PLP --> PL
CP --> PCo
PCoD --> PCo
PCoK --> PCo
PCo --> PS
PSC --> PS
PSIT --> PS
PSPR --> PS
CML[Common Messages Library]
PL --> CML
DRS[Device Registry Service]
CML --> DRS
PS --> DRS
PCe --> DRS
DDS[Device Data Service]
CML --> DDS
PS --> DDS
DCS[Device Configuration Service]
CML --> DCS
PS --> DCS
DAS[Device API Service]
CML --> DAS
PS --> DAS
PCe --> DAS
RAS[Registry API Service]
CML --> RAS
PS --> RAS
PCe --> RAS
AS[Account Service]
CML -.-> AS
PS -.-> AS
PCe -.-> AS
MUS[Management UI Service]
CML -.-> MUS
PS -.-> MUS
PCe -.-> MUS
```

48
doc/docker_database.md Normal file
View File

@@ -0,0 +1,48 @@
# Hlæja databases
## InfluxDB
InfluxDB is a high-performance time series database designed to handle large volumes of time-stamped data. It is commonly used for monitoring, analytics, and IoT applications, where data points are associated with timestamps (e.g., sensor readings, system metrics).
### Environment
```text
DOCKER_INFLUXDB_INIT_MODE: setup
DOCKER_INFLUXDB_INIT_USERNAME: influx
DOCKER_INFLUXDB_INIT_PASSWORD: password
DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: influxdbToken==
DOCKER_INFLUXDB_INIT_ORG: hlaeja_ltd
DOCKER_INFLUXDB_INIT_BUCKET: device-data
```
## PostgreSQL
PostgreSQL is a powerful, open-source relational database management system (RDBMS). Known for its reliability and advanced features, it supports SQL for querying and managing data, along with extensive functionality for scalability and extensibility.
### Environment
```text
POSTGRES_USER: postgres
POSTGRES_PASSWORD : password
```
## Apache Cassandra
Apache Cassandra is a distributed NoSQL database designed for handling large amounts of data across many commodity servers with no single point of failure. It is optimized for high availability, scalability, and fault tolerance.
### Environment
```text
CASSANDRA_USER: cassandra
CASSANDRA_PASSWORD: password
```
## Redis
Redis is an in-memory data store that can be used as a database, message broker, or cache layer. It is designed for high performance and low latency, making it suitable for real-time web applications.
### Environment
```text
REDIS_PASSWORD: password
```

75
doc/docker_hlaeja.md Normal file
View File

@@ -0,0 +1,75 @@
# Hlæja Services
## Device Data
### Environment
```text
SPRING_PROFILES_ACTIVE: docker
INFLUXDB_TOKEN: influxdbToken==
```
## Device Registry
### Environment
```text
SPRING_R2DBC_URL: r2dbc:postgresql://localhost:5432/device_registry
SPRING_R2DBC_USERBAME: services
SPRING_R2DBC_PASSWORD: password
JWT_PRIVATE_KEY: cert/private_key.pem
```
### Volume
Mount a local private key into the container. Read [`rsa_key.md`](./rsa_key.md) for how to generate.
```text
volumes:
- ./cert/device_private_key.pem:/app/resources/cert/private_key.pem
```
## Device API
### Environment
```text
SPRING_PROFILES_ACTIVE: docker
```
### Volume
Mount a local public key into the container. Read [rsa_key.md](./rsa_key.md) for how to generate.
Mount a local keystore into the container. Read [keystore.md](./keystore.md) for how to generate.
```text
volumes:
- ./cert/device_public_key.pem:/app/resources/cert/public_key.pem
- ./cert/device_api_keystore.p12:/app/resources/cert/keystore.p12
```
## Device Configuration
### Environment
```text
SPRING_PROFILES_ACTIVE: docker
```
## Registry API
### Environment
```text
SPRING_PROFILES_ACTIVE: docker
```
### Volume
Mount a local keystore into the container. Read [keystore.md](./keystore.md) for how to generate.
```text
volumes:
- ./cert/registry_api_keystore.p12:/app/resources/cert/keystore.p12
```

26
doc/global_settings.md Normal file
View File

@@ -0,0 +1,26 @@
# Global settings
Hlaeja services utilize Gradle options or environment variables to configure development settings, ensuring our configurations remain organized and secure.
## Overview
By using these methods, we can easily manage access to restricted resources and maintain a consistent configuration across our services. This approach enables us to keep sensitive information separate from our codebase.
## Gradle properties
To access repositories that require authentication, we set `repository.user` and `repository.token` properties in the `gradle.properties` file. To do this:
1. Open or create the `gradle.properties` file in your Gradle user home directory:
- On Unix-like systems (Linux, macOS), this is typically located at `~/.gradle/`.
- On Windows, this is typically located at `C:\Users\<YourUsername>\.gradle\`.
2. Add the following settings to the `gradle.properties` file:
```properties
repository.user=your_user
repository.token=your_token_value
```
## Environment variables
Alternatively, you can use `REPOSITORY_USER` and `REPOSITORY_TOKEN` environment variables to pass credentials to the application. These variables can be set in your system environment or through your IDE.

46
doc/keystore.md Normal file
View File

@@ -0,0 +1,46 @@
# Generate Keystore
### Generate Keystores
To generate a keystore for our API's and web manager, which is used to enable HTTPS, you can use the following command:
```shell
keytool -genkeypair -alias <service> -keyalg RSA -keysize 2048 -validity 3650 -dname "CN=<domain>" -keypass <password> -keystore ./cert/<keystore>.p12 -storetype PKCS12 -storepass <password>
```
This command generates a keystore with the following properties:
* \<service>: The alias for the service (e.g. device-api)
* \<domain>: The domain name for the service (e.g. deviceapi)
* \<password>: The password for the keystore and private key
* ./cert/\<keystore>.p12: The file path and name for the generated keystore
### Export the public certificate
Once you have generated a keystore, you can export the public certificate using the following command:
```shell
keytool -export -alias <service> -keystore ./cert/<keystore>.p12 -storepass <password> -file ./cert/<domain>.cer -rfc
```
This command exports the public certificate with the following properties:
* \<service>: The alias for the service (e.g. device-api)
* \<keystore>: The keystore file containing the private key and certificate (e.g. ./cert/deviceapi.p12)
* \<password>: The password for the keystore
* \<domain>: The domain name for the exported certificate file (e.g. deviceapi.cer)
* ./cert/\<domain>.cer: The file path and name for the exported public certificate
The exported public certificate is then used on devices to establish a secure connection to our API. Specifically, the certificate is installed on devices to enable them to trust our API's SSL/TLS connection, allowing for encrypted communication between the device and our API.
Note: Make sure to update your hosts file with an entry for the domain name (e.g. 127.0.0.1 deviceapi) to enable local development.
1. Open `hosts` file:
* On Unix-like systems (Linux, macOS), this directory is typically `/etc/hosts`.
* On Windows, this directory is typically `%SystemRoot%\System32\drivers\etc\hosts`.
2. Add the following lines to the `hosts` file:
```text
127.0.0.1 deviceapi # Hlæja Device API
```

0
IDENTITY.md → doc/rsa_key.md
View File

74
hlaeja-compose.yml Normal file
View File

@@ -0,0 +1,74 @@
name: hlaeja-development
networks:
develop:
name: develop
external: true
services:
device-registry:
image: hlaeja-device-registry:0.3.0-SNAPSHOT
container_name: DeviceRegistry
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
networks:
- develop
ports:
- 9010:8080
# mount a local `device_private_key.pem` into the container.
volumes:
- ./cert/device_private_key.pem:/app/resources/cert/private_key.pem
device-data:
image: hlaeja-device-data:0.1.1
container_name: DeviceData
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
INFLUXDB_TOKEN: influxdbToken==
networks:
- develop
ports:
- 9020:8080
device-configuration:
image: hlaeja-device-configuration:0.1.0
container_name: DeviceConfiguration
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
networks:
- develop
ports:
- 9030:8080
device-api:
image: hlaeja-device-api:0.2.0
container_name: DeviceApi
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
networks:
- develop
ports:
- 9000:8443
# mount a local `device_public_key.pem` and `device_api_keystore.p12` into the container.
volumes:
- ./cert/device_public_key.pem:/app/resources/cert/public_key.pem
- ./cert/device_api_keystore.p12:/app/resources/cert/keystore.p12
registry-api:
image: hlaeja-registry-api:0.1.0
container_name: RegistryApi
restart: unless-stopped
environment:
SPRING_PROFILES_ACTIVE: docker
networks:
- develop
ports:
- 9040:8443
# mount a local `registry_api_keystore.p12` into the container.
volumes:
- ./cert/registry_api_keystore.p12:/app/resources/cert/keystore.p12