ilum/core Docker 镜像 - 轩辕镜像
ilum/coreILUM Core: Spark orchestration platform for Kubernetes (REST API, Livy, HA)
1 收藏0 次下载activeilum镜像
Ilum Core
Enterprise Apache Spark orchestration platform for Kubernetes, enabling serverless data processing at scale.
 docker run -d \ --name ilum-core \ -p 9000:9000 \ -e MONGO_URI=mongodb://mongo:27017/ilum \ -e SPRING_KAFKA_BOOTSTRAP_SERVERS=kafka:9092 \ ilum/core:<version>
Note: For production deployments, use the official Ilum Helm Chart.
Supported Tags
<version>: Ilum 6.6.1 + Spark orchestration engine + Java 21
Environment Variables
Infrastructure (Required)
| Variable | Description |
|---|---|
| MONGO_URI | MongoDB connection string for metadata storage |
| SPRING_KAFKA_BOOTSTRAP_SERVERS | Kafka brokers for inter-service communication |
| ILUM_STORAGE_MINIO_URL | MinIO endpoint for Spark artifacts and logs |
| ILUM_STORAGE_MINIO_ACCESS_KEY | MinIO access key |
| ILUM_STORAGE_MINIO_SECRET_KEY | MinIO secret key |
Optional Configuration
| Variable | Default Value | Description |
|---|---|---|
| SERVER_PORT | 9000 | Ilum Core API port |
| ILUM_LIVY_PORT | 9888 | Apache Livy-compatible gateway port |
| ILUM_COMMUNICATION_TYPE | kafka | Communication mode: kafka or grpc |
NOTE:
Using grpc as communication type disables High Availability support. For production deployments with HA requirements, use kafka (default).
Data Persistence (Volumes)
Ilum Core is stateless. All persistent data is stored in external infrastructure components (MongoDB, Kafka, MinIO).
TIP: For disaster recovery, ensure regular backups of MongoDB (metadata) and MinIO (Spark artifacts, event logs).
Network Configuration
| Port | Service |
|---|---|
9000 | Ilum Core REST API + Apache Livy 0.8.0-compatible endpoints |
9888 | gRPC communication (ilum-core ↔ Spark jobs) |
Security
Non-root Architecture
The image runs as user ilum (UID 1001). Ensure that mounted volumes have appropriate permissions.
Secret Hygiene
CAUTION: Never pass sensitive credentials directly in environment variables!
Recommended methods:
- Kubernetes Secrets — mount MongoDB URI, Kafka credentials, and MinIO keys as secrets.
- IAM Roles (IRSA) — use cloud provider identity for storage access when running on managed Kubernetes.
Vulnerability Scanning
Before production deployment, scan the image:
bash# Trivy trivy image ilum/core:<version>
Building the Image
Run commands from the root of the ilum-backend repository:
bash# Build the service ./gradlew clean build # Build multi-platform Docker image ./gradlew dockerMultiplatformPush
Image Verification
Layer Analysis (Dive)
bashdive ilum/core:<version>
Software Bill of Materials (SBOM)
bash# Syft syft ilum/core:<version> -o spdx-json > sbom.json
Image Contents — Details
Dockerfile Components
- Base: Eclipse Temurin 21 JRE (via
ilum/java:21.0.9_10-jre) - Application: Spring Boot REST API (v6.6.1)
- Spark: Apache Spark 3.x binaries with cloud storage connectors
- Observability: OpenLineage 1.37.0 for data lineage
- Kyuubi Support: Kyuubi Spark SQL Engine 1.10.2 for serverless SQL
- Example Jobs: Pre-packaged PySpark and Scala job examples
Cloud Storage Connectors
- AWS S3: Hadoop AWS 3.3.4 + AWS SDK Bundle 1.12.262
- Google Cloud Storage: GCS Connector (Hadoop3) 2.2.13
- Azure Blob Storage: Hadoop Azure 3.3.4 + Azure Storage 7.0.1
API Documentation
The Ilum Core API is designed with an API-first approach.
Interactive API Documentation:
- Swagger UI:
http://<ilum-core>:9000/docs/api/ - OpenAPI Spec: Available in the
ilum-api-definitionmodule
API Version Compatibility:
| API Version | Ilum Version |
|---|---|
| 1.0.0 | <= 6.6.1 |
Architecture & Features
Job Types
Interactive Jobs
Long-running Spark applications that accept consecutive workload requests without re-initialization overhead. Ideal for:
- Real-time data exploration
- Multi-tenant query workloads
- Serverless SQL via Kyuubi
Single Jobs
Traditional batch Spark jobs submitted via API. Supports:
- Scheduled ETL pipelines
- One-off data transformations
- Integration with external orchestrators (Airflow, Argo Workflows)
Communication Modes
| Mode | Use Case | HA Support |
|---|---|---|
| Kafka | Production deployments requiring scalability and HA | ✅ Yes |
| gRPC (default) | Development/testing environments, simplified setup | ❌ No |
Cluster Support
- Kubernetes: Primary target for production workloads
- YARN: Legacy Hadoop clusters
- Local: Development and testing only
Infrastructure Dependencies
Ilum Core requires the following components to operate:
- MongoDB: Metadata storage (clusters, jobs, sessions)
- Apache Kafka: Event streaming and inter-service communication (when using Kafka mode)
- MinIO: S3-compatible storage for Spark JARs, Python archives, configuration files, and logs
All dependencies are automatically deployed when using the ilum-aio Helm chart.
Support and Feedback
- Issues: Bug Tracker
- Documentation: Ilum Docs
- Slack: Join our Community
pandoc/core
by pandoc
官方pandoc镜像，包含图像转换工具和pandoc-crossref过滤器，适用于通用文档转换任务。
441M+ pulls
上次更新：1 个月前
openstates/core
by Open States
暂无描述
500K+ pulls
上次更新：25 天前
clearandfizzy/core
by clearandfizzy
Clearandfizzy核心Docker容器，提供基础核心功能支持，作为构建或运行相关应用的基础组件。
5M+ pulls
上次更新：5 个月前
owncloudci/core
by owncloudci
用于使用核心测试应用程序的Drone插件
1M+ pulls
上次更新：3 个月前
fireflyiii/core
by fireflyiii
Firefly III 是一款开源的个人财务管理工具，支持用户详细跟踪收入、支出、预算及财务目标，具备多货币与多账户管理功能，数据本地存储以保障隐私安全，界面友好且可自定义，能帮助用户全面掌控个人财务状况，适合希望自主管理财务、记录每笔交易并规划长期财务目标的用户使用。
12310M+ pulls
上次更新：9 天前
odota/core
by odota
暂无描述
5M+ pulls
上次更新：5 天前
轩辕镜像配置手册
探索更多轩辕镜像的使用方法，找到最适合您系统的配置方式
登录仓库拉取
通过 Docker 登录认证访问私有仓库
Linux
在 Linux 系统配置镜像服务
Windows/Mac
在 Docker Desktop 配置镜像
Docker Compose
Docker Compose 项目配置
K8s Containerd
Kubernetes 集群配置 Containerd
K3s
K3s 轻量级 Kubernetes 镜像加速
宝塔面板
在宝塔面板一键配置镜像
群晖
Synology 群晖 NAS 配置
飞牛
飞牛 fnOS 系统配置镜像
极空间
极空间 NAS 系统配置服务
爱快路由
爱快 iKuai 路由系统配置
绿联
绿联 NAS 系统配置镜像
威联通
QNAP 威联通 NAS 配置
Podman
Podman 容器引擎配置
Singularity/Apptainer
HPC 科学计算容器配置
其他仓库配置
ghcr、Quay、nvcr 等镜像仓库
专属域名拉取
无需登录使用专属域名
需要其他帮助？请查看我们的 常见问题Docker 镜像访问常见问题解答 或 提交工单
镜像拉取常见问题
轩辕镜像免费版与专业版有什么区别？
免费版仅支持 Docker Hub 访问，不承诺可用性和速度；专业版支持更多镜像源，保证可用性和稳定速度，提供优先客服响应。
轩辕镜像支持哪些镜像仓库？
专业版支持 docker.io、gcr.io、ghcr.io、registry.k8s.io、nvcr.io、quay.io、mcr.microsoft.com、docker.elastic.co 等；免费版仅支持 docker.io。
流量耗尽错误提示
当返回 402 Payment Required 错误时，表示流量已耗尽，需要充值流量包以恢复服务。
410 错误问题
通常由 Docker 版本过低导致，需要升级到 20.x 或更高版本以支持 V2 协议。
manifest unknown 错误
先检查 Docker 版本，版本过低则升级；版本正常则验证镜像信息是否正确。
镜像拉取成功后，如何去掉轩辕镜像域名前缀？
使用 docker tag 命令为镜像打上新标签，去掉域名前缀，使镜像名称更简洁。
用户好评
来自真实用户的反馈，见证轩辕镜像的优质服务
oldzhang
运维工程师
Linux服务器
5
"Docker访问体验非常流畅，大镜像也能快速完成下载。"