Introduction

warning

interLink is in early development phase, thus subject to breaking changes with no guarantee of backward compatibility.

Targets

• K8s applications with tasks to be executed on HPC systems: This target focuses on Kubernetes applications that require high-performance computing (HPC) resources for executing tasks. These tasks might involve complex computations, simulations, or data processing that benefit from the specialized hardware and optimized performance of HPC systems.

• Remote "runner"-like application for heavy payload execution requiring GPUs: This target is designed for applications that need to execute heavy computational payloads, particularly those requiring GPU resources. These applications can be run remotely, leveraging powerful GPU hardware to handle tasks such as machine learning model training, data analysis, or rendering.

• Lambda-like functions calling on external resources: This target involves running containers on demand with specific computing needs. Now these resources might also be outside of the Kubernetes cluster thanks to interLink functionality.

Target providers

Our solution is designed to target a wide range of providers with container execution capabilities, including but not limited to:

• SLURM or HTCondor batch systems with Apptainer, Enroot, or Singularity: These batch systems are widely used in high-performance computing environments to manage and schedule jobs. By integrating with container runtimes like Apptainer, Enroot, or Singularity, our solution can efficiently execute containerized tasks on these systems.
• Remote/on-demand virtual machines with any container runtime: This includes virtual machines that can be provisioned on-demand and support container runtimes such as Docker, Podman, or others. This flexibility allows for scalable and dynamic resource allocation based on workload requirements.
• Remote Kubernetes clusters: Our solution can extend the capabilities of existing Kubernetes clusters, enabling them to offload workloads to another remote cluster. This is particularly useful for distributing workloads across multiple clusters for better resource utilization and fault tolerance.
• Lambda-like services: These are serverless computing services that execute code in response to events and automatically manage the underlying compute resources. By targeting these services, our solution can leverage the scalability and efficiency of serverless architectures for containerized workloads. All of this, while exposing a bare Kubernetes API kind of orchestration.

NOT a target

• Long-running services: Our solution is not designed for services that need to run continuously for extended periods. It is optimized for tasks that have a defined start and end, rather than persistent services exposing intra-cluster communication endpoints.
• Kubernetes Federation: We do not aim to support Kubernetes Federation, which involves managing multiple Kubernetes clusters as a single entity. Our focus is on enabling Kubernetes pods to execute on remote resources, not on federating all kind of resources on multiple clusters.

Deployment scenarios

In-cluster mode

This scenario involves deploying a Virtual Kubelet along with the interLink API server and the plugin to interact with a remote API. This setup allows Kubernetes pods to be executed on remote resources while all other components sits inside the Kubernetes cluster.

Service remote edge node

In this scenario, the Virtual Kubelet communicates with remote services deployed on a dedicate edge node exposing authenticated interLink APIs and its associated plugin. This setup is ideal for scenarios where edge computing resources are utilized for controlled communication b/w the Kubernetes cluster and the remote resources.

Tunneled mode

This deployment involves the Virtual Kubelet connecting to a remote interLink API server and its plugin through a secure tunnel. This setup ensures secure communication between the Kubernetes cluster and the remote resources, making it suitable for environments with strict security requirements or to host services on a multi user host like a login node.

For more information visit the architecture page