diff --git a/README.md b/README.md index 2f20f4f3..621915ae 100644 --- a/README.md +++ b/README.md @@ -53,6 +53,7 @@ If you have a spare domain name you can configure applications to be accessible * [Komga](https://komga.org/) - a media server for your comics, mangas, BDs and magazines * [Krusader](https://krusader.org/) - Twin panel file management for your desktop * [Lidarr](https://github.com/lidarr/Lidarr) - Music collection manager for Usenet and BitTorrent users +* [Loki](https://grafana.com/oss/loki/) - Loki is a horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus. * [Mealie](https://hay-kot.github.io/mealie/) - A self-hosted recipe manager and meal planner * [Minecraft Server](https://www.minecraft.net/) - Server edition of the popular building and exploring game * [MiniDLNA](https://sourceforge.net/projects/minidlna/) - simple media server which is fully compliant with DLNA/UPnP-AV clients @@ -79,6 +80,7 @@ If you have a spare domain name you can configure applications to be accessible * [Plex](https://www.plex.tv/) - Plex Media Server * [Portainer](https://portainer.io/) - for managing Docker and running custom images * [Prometheus](https://prometheus.io/) - Time series database and monitoring system (via stats role). +* [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/) - Promtail is an agent which ships the contents of local logs to a private Grafana Loki instance * [Prowlarr](https://github.com/Prowlarr/Prowlarr) - Indexer aggregator for Sonarr, Radarr, Lidarr, etc. * [pyLoad](https://pyload.net/) - A download manager with a friendly web-interface * [PyTivo](http://pytivo.org) - An HMO and GoBack server for TiVos. @@ -107,6 +109,12 @@ If you have a spare domain name you can configure applications to be accessible * [YouTubeDL-Material](https://github.com/Tzahi12345/YoutubeDL-Material) - Self-hosted YouTube downloader built on Material Design * [ZNC](https://wiki.znc.in/ZNC) - IRC bouncer to stay connected to favourite IRC networks and channels +## Preconfigured Application Stacks + +Ansible-NAS application [stacks](https://ansible-nas.io/docs/category/stacks/) are a number of applications deployed together and preconfigured to perform a common goal. + +* [Logging](https://ansible-nas.io/docs/applications/stacks/logging/) - application logging capture and search service based on Grafana Loki. + ## What This Could Do Ansible-NAS can run anything that's in a Docker image, which is why Portainer is included. A NAS configuration is a pretty personal thing based on what you download, what media you view, how many photos you take...so it's difficult to please everyone. @@ -115,7 +123,7 @@ That said, if specific functionality you want isn't included and you think other ## What This Doesn't Do -Ansible NAS doesn't set up your disk partitions, primarily because getting it wrong can be incredibly destructive. That aside, configuring partitions is usually a one-time (or very infrequent) event, so there's not much to be gained by automating it. Check out the [docs](https://davestephens.github.io/ansible-nas) for recommended setups. +Ansible NAS doesn't set up your disk partitions, primarily because getting it wrong can be incredibly destructive. That aside, configuring partitions is usually a one-time (or very infrequent) event, so there's not much to be gained by automating it. Check out the [docs](https://ansible-nas.io/docs/) for recommended setups. ## Installation @@ -138,7 +146,7 @@ Read the [migrating from FreeNAS](https://ansible-nas.io/docs/further-configurat Getting help is easy! You can: -* Read the [docs](https://davestephens.github.io/ansible-nas) +* Read the [docs](https://ansible-nas.io/docs/) * Start a [discussion](https://github.com/davestephens/ansible-nas/discussions) * Raise an [issue](https://github.com/davestephens/ansible-nas/issues) if you think you've found a bug * Chat on [Gitter](https://gitter.im/Ansible-NAS/Chat) diff --git a/nas.yml b/nas.yml index 2871de25..742a9bf4 100644 --- a/nas.yml +++ b/nas.yml @@ -46,6 +46,15 @@ - ansible-nas-docker - ansible-nas + ### + ### Stacks + ### + + - role: logging + tags: + - logging + + ### ### Applications ### @@ -193,6 +202,10 @@ tags: - lidarr + - role: loki + tags: + - loki + - role: mealie tags: - mealie @@ -297,6 +310,10 @@ tags: - prowlarr + - role: promtail + tags: + - promtail + - role: pyload tags: - pyload diff --git a/roles/logging/defaults/main.yml b/roles/logging/defaults/main.yml new file mode 100644 index 00000000..8dfb9955 --- /dev/null +++ b/roles/logging/defaults/main.yml @@ -0,0 +1,2 @@ +--- +logging_stack_enabled: false diff --git a/roles/logging/tasks/main.yml b/roles/logging/tasks/main.yml new file mode 100644 index 00000000..c6b96ab7 --- /dev/null +++ b/roles/logging/tasks/main.yml @@ -0,0 +1,29 @@ +--- +- name: Start logging stack + block: + - name: Enable logging roles + ansible.builtin.set_fact: + grafana_enabled: true + minio_enabled: true + loki_enabled: true + promtail_enabled: true + when: logging_stack_enabled is true + # vars: + # minio_enabled: true + # loki_enabled: true + # promtail_enabled: true + + +- name: Stop logging stack + block: + - name: Disable logging roles + ansible.builtin.set_fact: + grafana_enabled: false + minio_enabled: false + loki_enabled: false + promtail_enabled: false + when: logging_stack_enabled is false + # vars: + # minio_enabled: false + # loki_enabled: false + # promtail_enabled: false diff --git a/roles/loki/defaults/main.yml b/roles/loki/defaults/main.yml new file mode 100644 index 00000000..7342af5f --- /dev/null +++ b/roles/loki/defaults/main.yml @@ -0,0 +1,24 @@ +--- +# enable or disable the application +loki_enabled: false +loki_available_externally: false + +# directories +loki_data_directory: "{{ docker_home }}/loki" + +# network +loki_hostname: "loki" +loki_network_name: "loki" +loki_http_port: "3100" + +# docker +loki_container_name: "loki" +loki_image_name: "grafana/loki" +loki_image_version: "latest" + +# specs +loki_memory: "1g" + +# config +loki_log_retention: 672h +loki_log_level: warn diff --git a/roles/loki/handlers/main.yml b/roles/loki/handlers/main.yml new file mode 100644 index 00000000..8463e9fe --- /dev/null +++ b/roles/loki/handlers/main.yml @@ -0,0 +1,8 @@ +--- +- name: Restart Grafana + community.docker.docker_container: + name: grafana + image: grafana/grafana:latest + state: started + restart: true + listen: "restart grafana" diff --git a/roles/loki/tasks/main.yml b/roles/loki/tasks/main.yml new file mode 100644 index 00000000..8e565cf6 --- /dev/null +++ b/roles/loki/tasks/main.yml @@ -0,0 +1,78 @@ +--- +- name: Start Loki + block: + - name: Check for Minio installation + ansible.builtin.fail: + msg: "Loki requires Minio enabled and running for storage, please set that up first." + when: minio_enabled is false + + - name: Check for Grafana installation + ansible.builtin.fail: + msg: "Loki requires Grafana enabled and running for visualisation, please set that up first." + when: grafana_enabled is false + + - name: Include Stats variables + ansible.builtin.include_vars: ../../stats/defaults/main.yml + + - name: Create Loki Directories + ansible.builtin.file: + path: "{{ item }}" + state: directory + with_items: + - "{{ loki_data_directory }}" + + - name: Create Minio buckets for Loki + ansible.builtin.file: + path: "{{ item }}" + state: directory + with_items: + - "{{ minio_data_directory }}/data/loki-data" + - "{{ minio_data_directory }}/data/loki-ruler" + + - name: Template Loki config + ansible.builtin.template: + src: config.yml + dest: "{{ loki_data_directory }}/config.yml" + register: loki_config + + - name: Create loki Docker Container + community.docker.docker_container: + name: "{{ loki_container_name }}" + image: "{{ loki_image_name }}:{{ loki_image_version }}" + pull: true + command: "-config.file=/etc/loki/config.yml" + ports: + - "{{ loki_http_port }}:3100" + - 7946 + - 9095 + volumes: + - "{{ loki_data_directory }}/config.yml:/etc/loki/config.yml" + restart_policy: unless-stopped + memory: "{{ loki_memory }}" + restart: "{{ loki_config is changed }}" + labels: + traefik.enable: "{{ loki_available_externally | string }}" + traefik.http.routers.loki.rule: "Host(`{{ loki_hostname }}.{{ ansible_nas_domain }}`)" + traefik.http.routers.loki.tls.certresolver: "letsencrypt" + traefik.http.routers.loki.tls.domains[0].main: "{{ ansible_nas_domain }}" + traefik.http.routers.loki.tls.domains[0].sans: "*.{{ ansible_nas_domain }}" + traefik.http.services.loki.loadbalancer.server.port: "3100" + prometheus.io/scrape: "true" + prometheus.io/port: "3100" + prometheus.io/path: "/metrics" + + - name: Template Grafana Loki data source + ansible.builtin.template: + src: grafana-datasource.yml + dest: "{{ stats_grafana_config_directory }}/provisioning/datasources/loki.yml" + owner: "472" + notify: restart grafana + when: loki_enabled is true + +- name: Stop loki + block: + - name: Stop loki + community.docker.docker_container: + name: "{{ loki_container_name }}" + state: absent + when: loki_enabled is false diff --git a/roles/loki/templates/config.yml b/roles/loki/templates/config.yml new file mode 100644 index 00000000..9a99ff38 --- /dev/null +++ b/roles/loki/templates/config.yml @@ -0,0 +1,43 @@ +--- +server: + http_listen_port: {{ loki_http_port }} + log_level: {{ loki_log_level }} + +memberlist: + +schema_config: + configs: + - from: 2021-08-01 + store: boltdb-shipper + object_store: s3 + schema: v11 + index: + prefix: index_ + period: 24h + +common: + path_prefix: /loki + replication_factor: 1 + storage: + s3: + endpoint: "{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}:{{ minio_api_port }}" + insecure: true + bucketnames: loki-data + access_key_id: "{{ minio_admin_username }}" + secret_access_key: "{{ minio_admin_password }}" + s3forcepathstyle: true + ring: + kvstore: + store: memberlist + +ruler: + storage: + s3: + bucketnames: loki-ruler + +chunk_store_config: + max_look_back_period: {{ loki_log_retention }} + +table_manager: + retention_deletes_enabled: true + retention_period: {{ loki_log_retention }} \ No newline at end of file diff --git a/roles/loki/templates/grafana-datasource.yml b/roles/loki/templates/grafana-datasource.yml new file mode 100644 index 00000000..05916906 --- /dev/null +++ b/roles/loki/templates/grafana-datasource.yml @@ -0,0 +1,12 @@ +--- +apiVersion: 1 + +datasources: + - name: Loki + type: loki + access: proxy + url: http://{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}:{{ loki_http_port }} + jsonData: + httpHeaderName1: "X-Scope-OrgID" + secureJsonData: + httpHeaderValue1: "1" diff --git a/roles/promtail/defaults/main.yml b/roles/promtail/defaults/main.yml new file mode 100644 index 00000000..bdcb4799 --- /dev/null +++ b/roles/promtail/defaults/main.yml @@ -0,0 +1,23 @@ +--- +# enable or disable the application +promtail_enabled: false +promtail_available_externally: false + +# directories +promtail_data_directory: "{{ docker_home }}/promtail" + +# network +promtail_hostname: "promtail" +promtail_network_name: "promtail" +promtail_port: "9080" + +# docker +promtail_container_name: "promtail" +promtail_image_name: "grafana/promtail" +promtail_image_version: "latest" + +# specs +promtail_memory: "1g" + +# config +promtail_loki_url: http://{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}:{{ loki_http_port }}/loki/api/v1/push diff --git a/roles/promtail/tasks/main.yml b/roles/promtail/tasks/main.yml new file mode 100644 index 00000000..0b4dc228 --- /dev/null +++ b/roles/promtail/tasks/main.yml @@ -0,0 +1,54 @@ +--- +- name: Start Promtail + block: + - name: Check for Loki installation + ansible.builtin.fail: + msg: "Promtail requires Loki enabled and running as a write target, please set that up first." + when: loki_enabled is false + + - name: Include Loki variables + ansible.builtin.include_vars: ../../loki/defaults/main.yml + + - name: Create Promtail Directories + ansible.builtin.file: + path: "{{ item }}" + state: directory + with_items: + - "{{ promtail_data_directory }}/config" + - "{{ promtail_data_directory }}/data" + + - name: Template Promtail config + ansible.builtin.template: + src: config.yml + dest: "{{ promtail_data_directory }}/config/config.yml" + + - name: Create Promtail Docker Container + community.docker.docker_container: + name: "{{ promtail_container_name }}" + image: "{{ promtail_image_name }}:{{ promtail_image_version }}" + pull: true + command: "-config.file=/etc/promtail/config.yml" + ports: + - "{{ promtail_port }}:9080" + volumes: + - "{{ promtail_data_directory }}/config/config.yml:/etc/promtail/config.yml" + - "{{ promtail_data_directory }}/data/:/data" + - "/var/run/docker.sock:/var/run/docker.sock" + restart_policy: unless-stopped + memory: "{{ promtail_memory }}" + labels: + traefik.enable: "{{ promtail_available_externally | string }}" + traefik.http.routers.promtail.rule: "Host(`{{ promtail_hostname }}.{{ ansible_nas_domain }}`)" + traefik.http.routers.promtail.tls.certresolver: "letsencrypt" + traefik.http.routers.promtail.tls.domains[0].main: "{{ ansible_nas_domain }}" + traefik.http.routers.promtail.tls.domains[0].sans: "*.{{ ansible_nas_domain }}" + traefik.http.services.promtail.loadbalancer.server.port: "9080" + when: promtail_enabled is true + +- name: Stop promtail + block: + - name: Stop promtail + community.docker.docker_container: + name: "{{ promtail_container_name }}" + state: absent + when: promtail_enabled is false diff --git a/roles/promtail/templates/config.yml b/roles/promtail/templates/config.yml new file mode 100644 index 00000000..77cf9283 --- /dev/null +++ b/roles/promtail/templates/config.yml @@ -0,0 +1,22 @@ +--- +server: + http_listen_port: 9080 + grpc_listen_port: 0 + +positions: + filename: /data/positions.yml + +clients: + - url: {{ promtail_loki_url }} + tenant_id: 1 + +scrape_configs: + - job_name: docker_scrape + docker_sd_configs: + - host: unix:///var/run/docker.sock + refresh_interval: 5s + relabel_configs: + - source_labels: ['__meta_docker_container_name'] + regex: '/(.*)' + target_label: 'container' + diff --git a/website/docs/applications/observability/_category_.json b/website/docs/applications/observability/_category_.json new file mode 100644 index 00000000..3a878ef6 --- /dev/null +++ b/website/docs/applications/observability/_category_.json @@ -0,0 +1,7 @@ +{ + "label": "Observability", + "link": { + "type": "generated-index", + "description": "All of the observability tooling available for installation with Ansible-NAS." + } +} diff --git a/website/docs/applications/observability/grafana.md b/website/docs/applications/observability/grafana.md new file mode 100644 index 00000000..1d7a7304 --- /dev/null +++ b/website/docs/applications/observability/grafana.md @@ -0,0 +1,17 @@ +--- +title: "Grafana" +--- + +Homepage: + +Docker image: [Grafana](https://hub.docker.com/r/grafana/grafana) + +Query, visualize, alert on, and understand your data no matter where it’s stored. With Grafana you can create, explore, and share all of your data through beautiful, flexible dashboards. + +## Usage + +Set `stats_enabled: true` in your `inventories//group_vars/nas.yml` file. + +Grafana's web interface can be found at . + + diff --git a/website/docs/applications/stacks/_category_.json b/website/docs/applications/stacks/_category_.json new file mode 100644 index 00000000..6acc3d34 --- /dev/null +++ b/website/docs/applications/stacks/_category_.json @@ -0,0 +1,7 @@ +{ + "label": "Stacks", + "link": { + "type": "generated-index", + "description": "Application stacks install a number of different apps together to perform a common goal. There are all of the application stacks available for installation with Ansible-NAS." + } +} diff --git a/website/docs/applications/stacks/logging.md b/website/docs/applications/stacks/logging.md new file mode 100644 index 00000000..3eee7a76 --- /dev/null +++ b/website/docs/applications/stacks/logging.md @@ -0,0 +1,29 @@ +--- +title: "Logging" +--- + +The logging stack sets up a fully functional application logging capture and search service based on [Loki](https://grafana.com/oss/loki/), viewable via Grafana. + +To enable it, add the following to your `inventories//group_vars/nas.yml`: + +``` +logging_stack_enabled: true +``` + +Which is equivalent to: + +``` +minio_enabled: true +loki_enabled: true +promtail_enabled: true +grafana_enabled: true +``` + +Once set up, all container stdout logs will be captured and stored. You'll find the Loki data source available in Grafana. + +Read more: + + - [Grafana](../observability/grafana.md) + - [Loki](../system-tools/loki.md) + - [Minio](../system-tools/minio.md) + - [Promtail](../system-tools/promtail.md) diff --git a/website/docs/applications/system-tools/loki.md b/website/docs/applications/system-tools/loki.md new file mode 100644 index 00000000..4a2061a7 --- /dev/null +++ b/website/docs/applications/system-tools/loki.md @@ -0,0 +1,19 @@ +--- +title: "Loki" +--- + +Homepage: + +Docker Container: [Loki](https://hub.docker.com/r/grafana/loki) + +Loki is a log aggregation system designed to store and query logs from all your applications and infrastructure. + +## Usage + +Set `loki_enabled: true` in your `inventories//nas.yml` file. + +Loki doesn't have a web interface. To see what it's doing look at the container logs from your Ansible-NAS shell: + +``` +docker logs loki -f +``` diff --git a/website/docs/applications/system-tools/minio.md b/website/docs/applications/system-tools/minio.md index 942ae28f..a45cab12 100644 --- a/website/docs/applications/system-tools/minio.md +++ b/website/docs/applications/system-tools/minio.md @@ -1,5 +1,5 @@ --- -title: "Minioo" +title: "Minio" --- Homepage: diff --git a/website/docs/applications/system-tools/promtail.md b/website/docs/applications/system-tools/promtail.md new file mode 100644 index 00000000..c6682531 --- /dev/null +++ b/website/docs/applications/system-tools/promtail.md @@ -0,0 +1,21 @@ +--- +title: "Promtail" +--- + +Homepage: + +Docker image: [Promtail](https://hub.docker.com/r/grafana/promtail) + +Promtail is an agent which ships the contents of local logs to a private Grafana Loki instance or Grafana Cloud. It is usually deployed to every machine that has applications needed to be monitored. + +It primarily: + + - Discovers targets + - Attaches labels to log streams + - Pushes them to the Loki instance. + +## Usage + +Set `promtail_enabled: true` in your `inventories//nas.yml` file. + +To see what Promtail is doing (and what containers it has discovered for tailing), visit the web interface at at .