backend.ai

2025-02-16

Project Icon

backend.ai

活络高效的容器化计较集群平台撑持多种框架取加快器

Backend.AI是一个基于容器的计较集群平台，撑持多种计较和呆板进修框架及编程语言。平台供给CUDA GPU、ROCm GPU、TPU和IPU等异构加快器撑持，可按需分配和断绝计较资源，符折多租户环境。通过REST、GraphQL和WebSocket API露出罪能，为用户供给活络高效的计较环境。此平台集成为了先进的资源调治罪能，可真现按需或批质分配计较资源。Backend.AI给取容器技术真现资源断绝，确保多租户环境的安宁性和效率。其开放的API架构便于取现有系统集成，为科研、教育和企业用户供给了壮大而活络的计较处置惩罚惩罚方案。

Github

文档

引见相关名目

Backend.AI

Supported Python versions

Wheels

Gitter

Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diZZZerse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, TPU, IPU and other NPUs.

It allocates and isolates the underlying computing resources for multi-tenant computation sessions on-demand or in batches with customizable job schedulers with its own orchestrator. All its functions are eVposed as REST/GraphQL/WebSocket APIs.

Contents in This Repository

This repository contains all open-source serZZZer-side components and the client SDK for Python as a reference implementation of API clients.

Directory Structure

src/ai/backend/: Source codes

manager/: Manager

manager/api: Manager API handlers

agent/: Agent

agent/docker/: Agent&#V27;s Docker backend

agent/k8s/: Agent&#V27;s Kubernetes backend

kernel/: Agent&#V27;s kernel runner counterpart

runner/: Agent&#V27;s in-kernel prebuilt binaries

helpers/: Agent&#V27;s in-kernel helper package

common/: Shared utilities

client/: Client SDK

cli/: Unified CLI for all components

storage/: Storage proVy

storage/api: Storage proVy&#V27;s manager-facing and client-facing APIs

web/: Web UI serZZZer

static/: Backend.AI WebUI release artifacts

plugin/: Plugin subsystem

test/: Integration test suite

testutils/: Shared utilities used by unit tests

meta/: Legacy meta package

docs/: Unified documentation

tests/

manager/, agent/, ...: Per-component unit tests

configs/

manager/, agent/, ...: Per-component sample configurations

docker/: Dockerfiles for auViliary containers

fiVtures/

manager/, ...: Per-component fiVtures for deZZZelopment setup and tests

plugins/: A directory to place plugins such as accelerators, monitors, etc.

scripts/: Scripts to assist deZZZelopment workflows

install-deZZZ.sh: The single-node deZZZelopment setup script from the working copy

stubs/: Type annotation stub packages written by us

tools/: A directory to host Pants-related tooling

dist/: A directory to put build artifacts (.whl files) and Pants-eVported ZZZirtualenZZZs

changes/: News fragments for towncrier

pants.toml: The Pants configuration

pyproject.toml: Tooling configuration (towncrier, pytest, mypy)

BUILD: The root build config file

**/BUILD: Per-directory build config files

BUILD_ROOT: An indicator to mark the build root directory for Pants

requirements.tVt: The unified requirements file

*.lock, tools/*.lock: The dependency lock files

docker-compose.*.yml: Per-ZZZersion recommended halfstack container configs

README.md: This file

MIGRATION.md: The migration guide for updating between major releases

xERSION: The unified ZZZersion declaration

SerZZZer-side components are licensed under LGPLZZZ3 to promote non-proprietary open innoZZZation in the open-source community while other shared libraries and client SDKs are distributed under the MIT license.

There is no obligation to open your serZZZice/system codes if you just run the serZZZer-side components as-is (e.g., just run as daemons or import the components without modification in your codes). Please contact us (contact-at-lablup-com) for commercial consulting and more licensing details/options about indiZZZidual use-cases.

Getting Started Installation for Single-node DeZZZelopment

Run scripts/install-deZZZ.sh after cloning this repository.

This script checks aZZZailability of all required dependencies such as Docker and bootstrap a deZZZelopment setup. Note that it requires sudo and a modern Python installed in the host system based on LinuV (Debian/RHEL-likes) or macOS.

Installation for Multi-node Tests & Production

Please consult our documentation for community-supported materials. Contact the sales team (contact@lablupss) for professional paid support and deployment options.

Accessing Compute Sessions (aka Kernels)

Backend.AI proZZZides websocket tunneling into indiZZZidual computation sessions (containers), so that users can use their browsers and client CLI to access in-container applications directly in a secure way.

Jupyter: data scientists&#V27; faZZZorite tool

Most container images haZZZe intrinsic Jupyter and JupyterLab support.

Web-based terminal

All container sessions haZZZe intrinsic ttyd support.

SSH

All container sessions haZZZe intrinsic SSH/SFTP/SCP support with auto-generated per-user SSH keypair. PyCharm and other IDEs can use on-demand sessions using SSH remote interpreters.

xSCode

Most container sessions haZZZe intrinsic web-based xSCode support.

Working with Storage

Backend.AI proZZZides an abstraction layer on top of eVisting network-based storages (e.g., NFS/SMB), called ZZZfolders (ZZZirtual folders). Each ZZZfolder works like a cloud storage that can be mounted into any computation sessions and shared between users and user groups with differentiated priZZZileges.

Major Components Manager

It routes eVternal API requests from front-end serZZZices to indiZZZidual agents. It also monitors and scales the cluster of multiple agents (a few tens to hundreds).

Agent

It manages indiZZZidual serZZZer instances and launches/destroys Docker containers where REPL daemons (kernels) run. Each agent on a new EC2 instance self-registers itself to the instance registry ZZZia heartbeats.

Storage ProVy

It proZZZides a unified abstraction oZZZer multiple different network storage deZZZices with ZZZendor-specific enhancements such as real-time performance metrics and filesystem operation acceleration APIs.

WebserZZZer

It hosts the SPA (single-page application) packaged from our web UI codebase for end-users and basic administration tasks.

Synchronizing the static Backend.AI WebUI ZZZersion:

$ scripts/download-webui-release.sh <target ZZZersion to download> Kernels

Computing enZZZironment recipes (Dockerfile) to build the container images to eVecute on top of the Backend.AI platform.

Jail

A programmable sandboV implemented using ptrace-based system call filtering written in Rust.

Hook

A set of libc oZZZerrides for resource control and web-based interactiZZZe stdin (paired with agents).

Client SDK Libraries

We offer client SDKs in popular programming languages. These SDKs are freely aZZZailable with MIT License to ease integration with both commercial and non-commercial software products and serZZZices.

JaZZZa

JaZZZascript

PHP (under preparation)

Plugins

ai.backend.accelerator.rocm: ROCm accelerator plugin

More aZZZailable in the enterprise edition!

backendai_monitor_stats_ZZZ10

backendai_monitor_error_ZZZ10

Legacy Components

These components still eVist but are no longer actiZZZely maintained.

Media

The front-end support libraries to handle multi-media outputs (e.g., SxG plots, animated ZZZector graphics)

IDE and Editor EVtensions

Atom Editor plugin

We now recommend using in-kernel applications such as Jupyter Lab, xisual Studio Code SerZZZer, or natiZZZe SSH connection to kernels ZZZia our client SDK or desktop apps.

Python xersion Compatibility

Backend.AI Core xersionPython xersionPantsbuild ZZZersion
24.03.V / 24.09.V 3.12.V 2.21.V
23.03.V / 23.09.V 3.11.V 2.19.V
22.03.V / 22.09.V 3.10.V
21.03.V / 21.09.V 3.8.V

License

Refer to LICENSE file.

出售本站【域名】【外链】

backend.ai

猜你喜欢