AcademicResearch · containers & edge computing

Container virtualisation frameworks in Edge-to-Cloud and serverless environments

Bachelor’s thesis (9.5/10). Extended the DAGonStar workflow engine with Docker, Kubernetes (k3s/k0s), Nomad and Apptainer executors and a cross-runtime data-staging layer, then measured performance, overhead and energy on a real Edge–Fog–Cloud testbed.

Role
Author — design, implementation, experimentation and evaluation
Context
Universidad Carlos III de Madrid
Period
2025 – 2026
View source on GitHub(opens in a new tab)

01Overview

Edge-to-Cloud infrastructures mix ARM devices, desktop machines and managed cloud services, with no shared filesystem between them. My thesis extended DAGonStar — a DAG-based scientific workflow engine — so the same workflow can run its tasks in Docker, Kubernetes (k3s/k0s), Nomad or Apptainer, locally or on remote nodes, and then measured what each choice costs in time, resources and energy.

02Context

Bachelor’s thesis in Computer Engineering at Universidad Carlos III de Madrid (2025–2026), supervised by Dante Domizzi Sánchez Gallegos and awarded 9.5/10. The testbed combined a Fog node (desktop PC acting as orchestrator), an Edge node (Raspberry Pi 5 reading a DHT11 temperature/humidity sensor through an Arduino) and a Cloud tier (MongoDB Atlas).

03Problem statement

Distributed workflows across Edge, Fog and Cloud face three problems — dependency isolation, portability across heterogeneous hardware (x86_64 and ARMv8), and moving intermediate data between nodes that share no filesystem. Choosing a container runtime for constrained edge devices also has measurable consequences for latency, energy and hardware temperature that are rarely quantified.

04My responsibilities

  • Designed and implemented task classes for Apptainer, Docker, Kubernetes and Nomad, each with a remote variant, behind DAGonStar’s homogeneous task interface.
  • Built a cross-runtime staging mechanism (Stager and Data Mover) combining host-shared areas, SSH/SCP transfers and pod-to-pod streaming for Kubernetes.
  • Integrated Prometheus, Node Exporter and Grafana to monitor execution time, resource usage, temperature and energy.
  • Designed and ran single-node, distributed and full IoT-telemetry experiments, and analysed the results.

05Architecture overview

The Fog node runs DAGonStar and orchestrates tasks on the Edge node over SSH. On the Edge, the selected runtime — Docker, Apptainer, Nomad or k3s/k0s — executes the tasks that capture and process sensor data. The Stager/Data Mover moves intermediate files between tasks and nodes, results are persisted in MongoDB Atlas, and Node Exporter metrics are scraped by Prometheus and visualised in Grafana.

Architecture overviewEdge–Fog–Cloud testbed
Edge–Fog–Cloud testbedA Fog node runs the DAGonStar workflow engine and its Stager/Data Mover. It dispatches remote tasks over SSH to a Raspberry Pi 5 edge node running Docker, Apptainer, Nomad or k3s/k0s, with a DHT11 sensor attached via Arduino. Results are persisted to MongoDB Atlas in the cloud. Node Exporter metrics are scraped by Prometheus and visualised in Grafana.Fog — desktop PCEdge — Raspberry Pi 5CloudMonitoringSSHstagingpersisthost metricsscrapeDAGonStarDAG workflow engineStager · Data MoverSSH / SCP · pod streamingDockerApptainerNomadk3s · k0sDHT11 sensor via Arduinotemperature & humidityMongoDB Atlasmetrics & resultsNode ExporterPrometheusGrafana
  • Delivery & automation
  • Code & configuration
  • Platform / runtime
  • Device
  • Data & storage
  • Observability
Text description of the diagram

A Fog node runs the DAGonStar workflow engine and its Stager/Data Mover. It dispatches remote tasks over SSH to a Raspberry Pi 5 edge node running Docker, Apptainer, Nomad or k3s/k0s, with a DHT11 sensor attached via Arduino. Results are persisted to MongoDB Atlas in the cloud. Node Exporter metrics are scraped by Prometheus and visualised in Grafana.

Flows

  • DAGonStar → Docker (SSH)
  • Stager · Data Mover → Edge — Raspberry Pi 5 (staging)
  • DHT11 sensor via Arduino → Nomad
  • Apptainer → MongoDB Atlas (persist)
  • Edge — Raspberry Pi 5 → Node Exporter (host metrics)
  • Node Exporter → Prometheus (scrape)
  • Prometheus → Grafana

Components

DAGonStar · Stager · Data Mover · Docker · Apptainer · Nomad · k3s · k0s · DHT11 sensor via Arduino · MongoDB Atlas · Node Exporter · Prometheus · Grafana

06Technology stack

Runtimes & orchestrators
  • Docker
  • Apptainer
  • HashiCorp Nomad
  • k3s
  • k0s
Workflow & data
  • DAGonStar
  • Python
  • SSH / SCP
  • MongoDB Atlas
Observability
  • Prometheus
  • Node Exporter
  • Grafana
  • Matplotlib
Hardware
  • Raspberry Pi 5
  • Arduino + DHT11
  • x86_64 desktop

07Implementation approach

  • A class hierarchy of task types — ApptainerTask, DockerTask, KubernetesTask, NomadTask and their remote variants — so users switch runtime without changing how the workflow is written.
  • Credentials and connection parameters centralised in a configuration file (dagon.ini) rather than embedded in workflow code.
  • Volume-based staging for Docker, Apptainer and Nomad; streaming between pods for Kubernetes, removing the assumption of a shared filesystem.
  • Benchmarks implemented as DAGonStar workflows, from edge-only runs to distributed Edge–Fog execution and a complete Edge–Fog–Cloud IoT telemetry workflow.
  • Energy, CPU, RAM and temperature monitored in real time with Grafana dashboards per runtime, and post-processed with Python and Matplotlib.

08Challenges & engineering decisions

  1. Challenge

    No shared filesystem between Edge, Fog and Cloud nodes.

    Decision

    A transversal staging layer that picks the mechanism per runtime — shared host volumes, SSH/SCP, or pod-to-pod streaming.

  2. Challenge

    Comparing runtimes fairly when they processed different amounts of work (Nomad handled ~150 records, Docker and Apptainer ~292).

    Decision

    Normalise energy by workload (joules per record) instead of comparing raw energy totals.

  3. Challenge

    Kubernetes (k3s/k0s) on a Raspberry Pi 5 with an 8 GB SD card became unstable during long runs as cluster components and images filled the disk.

    Decision

    Keep the KubernetesTask implementation (fully functional) but report the hardware limitation as a finding, and support lighter runtimes or delegation to the Fog tier.

09Security considerations

  • Remote execution and data transfers run over SSH/SCP rather than unauthenticated channels.
  • Connection credentials kept in a dedicated configuration file, separate from workflow definitions.
  • Identified as future work — end-to-end encryption in staging and secret vaults with automatic rotation for regulated data.

10Operational considerations

  • Thermal impact matters on the edge — Nomad’s higher CPU use (10.8% mean vs 0.38% for Apptainer) raised operating temperature by about 5 °C, a throttling risk in long-running or hot deployments.
  • Apptainer’s efficiency depends on pre-built images and active storage management; Docker and Nomad handle image distribution more transparently at a higher baseline cost.

11Results & impact

  • Apptainer was the most efficient runtime on the edge node — 1.68 J per record, with under 1% orchestration overhead.
  • Apptainer reduced energy consumption by 26% compared with Docker and 43% compared with Nomad.
  • Docker offered a balanced profile for heterogeneous Edge–Fog deployments; Nomad suits centralised multi-node orchestration but not low-latency, energy-constrained edge workloads.
  • Kubernetes executors are functional, but k3s/k0s proved unviable on minimal edge hardware — a documented finding with future work on multi-Pi or HPC clusters.
Normalised efficiency on the edge node — full IoT telemetry workflow
RuntimeRecordsEnergy (J)J / recordTemp. (°C)Overhead
Apptainer292490.81.6846.1< 1%
Docker291666.32.2947.3~ 29.2%
Nomad150859.35.7351.154.4%

Source — thesis, Table 4.15.

12Lessons learned

  • Normalise before you compare — raw totals hid that Nomad spent most of its energy on orchestration rather than useful work.
  • Runtime choice is an infrastructure decision with energy, thermal and hardware-lifetime consequences, not only a developer preference.
  • Heterogeneous infrastructure needs an engine that can pick the lightest viable runtime per node, or delegate work upstream.