!10 [Feature][Fix][Doc] Tencent NAT Auto-Portforwarding
* [Fix] Relocate User Permanent SSH Public Key queries to DevcontainerService Layer * [Fix] Add Unix Timestamps in DB table `devstar_devcontainer` * [Feature] Tencent NAT port forwarding * [Doc] k8s Operator RBAC: ServiceAccount, ClusterRole, ClusterRoleBinding, etc. * [fix] k8s Operator Reconciler error while converting YAML to JSON * [Doc] Added DevStar API Doc * [fix] detailed errors while listing user devcontainers * [fix] Invalid metadata.labels: value must be no more than 63 characters
This commit is contained in:
182
README_ZH.md
182
README_ZH.md
@@ -153,6 +153,10 @@ data:
|
||||
devstar.ssh_key_pair: |
|
||||
KEY_SIZE = <写入希望生成的SSH密钥长度,比如 4096 ,默认值 2048>
|
||||
|
||||
devstar.cloud: |
|
||||
ENABLED = true
|
||||
PROVIDER = tencent
|
||||
|
||||
```
|
||||
|
||||
对于 `Secrets` 类型,参考:
|
||||
@@ -170,6 +174,17 @@ stringData:
|
||||
WECHAT_OFFICIAL_ACCOUNT_APP_SECRET=<微信公众号SECRET>
|
||||
WECHAT_OFFICIAL_ACCOUNT_MESSAGE_TOKEN = <微信公众号自定义Token>
|
||||
WECHAT_OFFICIAL_ACCOUNT_MESSAGE_AES_KEY = <微信公众号AES加密密钥>
|
||||
|
||||
devstar.cloud.tencent: |
|
||||
ENDPOINT = <API访问端点名称,例如 vpc.tencentcloudapi.com>
|
||||
REGION = <区域代码,例如 ap-shanghai>
|
||||
NAT_GATEWAY_ID = <腾讯云控制台使用的 NAT网关 ID>
|
||||
PUBLIC_IP_ADDRESS = <公网IP>
|
||||
PRIVATE_IP_ADDRESS = <内网IP>
|
||||
IP_PROTOCOL = TCP
|
||||
SECRET_ID = <腾讯云密钥ID>
|
||||
SECRET_KEY = <腾讯云密钥内容>
|
||||
|
||||
```
|
||||
|
||||
为了使 Gitea启动时,能将配置信息同步到 `app.ini` 文件中,需要在 Helm Charts `/values.yaml` 中的 `additionalConfigSources` 中挂载创建的 `ConfigMaps` 与 `Secrets` 资源,示例格式:
|
||||
@@ -187,52 +202,169 @@ stringData:
|
||||
|
||||
为了能够在运行时创建 Dev Container,需要配置**基于角色的访问控制**(Role-Based Access Control, RBAC)。
|
||||
|
||||
由于 DevStar Studio 运行后将自动创建 ServiceAccount `default`:
|
||||
首先需要再 DevStar Studio 的 Helm Chart 中指定名称为 `devstar-studio-gitea-serviceaccount` 的 ServiceAccount
|
||||
|
||||
修改 `values.yaml` 文件内容如下:
|
||||
|
||||
```bash
|
||||
kubectl get serviceaccounts -n devstar-studio-ns
|
||||
# NAME SECRETS AGE
|
||||
# default 1 44d
|
||||
# devstar-studio-postgresql 1 42d
|
||||
## @section ServiceAccount
|
||||
|
||||
## @param serviceAccount.create Enable the creation of a ServiceAccount
|
||||
## @param serviceAccount.name Name of the created ServiceAccount, defaults to release name. Can also link to an externally provided ServiceAccount that should be used.
|
||||
## @param serviceAccount.automountServiceAccountToken Enable/disable auto mounting of the service account token
|
||||
## @param serviceAccount.imagePullSecrets Image pull secrets, available to the ServiceAccount
|
||||
## @param serviceAccount.annotations Custom annotations for the ServiceAccount
|
||||
## @param serviceAccount.labels Custom labels for the ServiceAccount
|
||||
serviceAccount:
|
||||
create: true
|
||||
name: "devstar-studio-gitea-serviceaccount"
|
||||
automountServiceAccountToken: true
|
||||
imagePullSecrets: []
|
||||
# - name: private-registry-access
|
||||
annotations: {}
|
||||
labels: {}
|
||||
|
||||
```
|
||||
|
||||
因此,只需要创建关联该 ServiceAccount 的 RoleBinding,在 Helm Charts 目录下 创建文件夹 `templates/devstar-devcontainer-rbac/`
|
||||
当 Helm Chart 部署成功后,可通过下列命令查看 `devstar-studio-gitea-serviceaccount` 的实际内容:
|
||||
|
||||
```bash
|
||||
kubectl get serviceaccount -n devstar-studio-ns devstar-studio-gitea-serviceaccount -o yaml
|
||||
```
|
||||
|
||||
1. 创建 Role
|
||||
```yaml
|
||||
kind: Role
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
apiVersion: v1
|
||||
automountServiceAccountToken: true
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: read-devcontainer-pods
|
||||
namespace: {{ .Values.devstar.devcontainer.namespace }}
|
||||
rules:
|
||||
- apiGroups: [""] # meaning: the core API Group
|
||||
resources: ["pods"]
|
||||
verbs: ["get", "watch", "list"]
|
||||
annotations:
|
||||
meta.helm.sh/release-name: devstar-studio
|
||||
meta.helm.sh/release-namespace: devstar-studio-ns
|
||||
creationTimestamp: "2024-10-01T06:49:56Z"
|
||||
labels:
|
||||
app: gitea
|
||||
app.kubernetes.io/instance: devstar-studio
|
||||
app.kubernetes.io/managed-by: Helm
|
||||
app.kubernetes.io/name: gitea
|
||||
app.kubernetes.io/version: rootless-dev-1fa57bca2dc6922b093157e17f16d5652de1420d
|
||||
helm.sh/chart: gitea-0.0.0
|
||||
version: rootless-dev-1fa57bca2dc6922b093157e17f16d5652de1420d
|
||||
name: devstar-studio-gitea-serviceaccount
|
||||
namespace: devstar-studio-ns
|
||||
resourceVersion: "92197844"
|
||||
uid: 71960972-c76e-4b6a-98b5-9f66f90aefca
|
||||
secrets:
|
||||
- name: devstar-studio-gitea-serviceaccount-token-d499q
|
||||
```
|
||||
|
||||
2. 创建 RoleBiding
|
||||
|
||||
下一步,需要创建关联该 ServiceAccount 的 RoleBinding
|
||||
|
||||
1. ClusterRole
|
||||
|
||||
通过运行命令查看编辑 DevcontainerApp 的 ClusterRole
|
||||
|
||||
```bash
|
||||
kubectl get clusterrole devcontainer-operator-devcontainerapp-editor-role -o yaml
|
||||
```
|
||||
|
||||
显示结果如下
|
||||
|
||||
```yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: read-secrets
|
||||
namespace: {{ .Values.devstar.devcontainer.namespace }}
|
||||
labels:
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
app.kubernetes.io/name: devcontainer-operator
|
||||
name: devcontainer-operator-devcontainerapp-editor-role
|
||||
rules:
|
||||
- apiGroups:
|
||||
- devcontainer.devstar.cn
|
||||
resources:
|
||||
- devcontainerapps
|
||||
verbs:
|
||||
- create
|
||||
- delete
|
||||
- get
|
||||
- list
|
||||
- patch
|
||||
- update
|
||||
- watch
|
||||
- apiGroups:
|
||||
- devcontainer.devstar.cn
|
||||
resources:
|
||||
- devcontainerapps/status
|
||||
verbs:
|
||||
- get
|
||||
|
||||
```
|
||||
|
||||
上述结果表明,ClusterRole `devcontainer-operator-devcontainerapp-editor-role` 具有如下权限:
|
||||
- 对于 k8s CRD 资源 `DevcontainerApp`:创建、删除、获取、列举、部分更新、全量更新、监听
|
||||
- 对于 k8s CRD 资源 `DevcontainerApp` 的 status 域: 获取
|
||||
|
||||
|
||||
2. 创建 ClusterRoleBinding
|
||||
|
||||
在 Helm Charts 目录下 创建文件夹 `templates/devstar-devcontainer-rbac/`,创建下列 ClusterRoleBinding
|
||||
|
||||
```yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: devcontainer-operator
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
name: devstar-devcontainerapps-editor-role-binding
|
||||
roleRef:
|
||||
kind: Role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
name: admin
|
||||
namespace:
|
||||
kind: ClusterRole
|
||||
name: devcontainer-operator-devcontainerapp-editor-role
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: User
|
||||
name: admin
|
||||
- kind: ServiceAccount
|
||||
name: {{ include "gitea.serviceAccountName" . }}
|
||||
namespace: {{ .Release.Namespace | quote }}
|
||||
|
||||
```
|
||||
|
||||
**注意**: RBAC 应当对 Gitea 透明,即不得将 RBAC 信息挂载到 `/values.yaml` 中如 1.2.2 小节所示的 `additionalConfigSources` 字段
|
||||
|
||||
至此,DevStar Studio 已被赋予 k8s CRD 资源 `DevcontainerApp` 的操作权限,即可操作集群,创建或修改 DevContainer
|
||||
|
||||
> **补充**: DevStar Studio 获取集群的控制权限
|
||||
>
|
||||
> ```go
|
||||
> // GetKubernetesClient
|
||||
> func GetKubernetesClient(ctx *context.Context) (dynamicclient.Interface, error) {
|
||||
>
|
||||
> // 1. 尝试从集群外获取 kubectl 配置信息
|
||||
> config, err := clientcmd.BuildConfigFromFlags("", clientgocmdtools.RecommendedHomeFile)
|
||||
> if err != nil {
|
||||
> // 1.1 集群外尝试失败,改从集群内获取 kubectl 配置信息
|
||||
> log.Warn("Failed to obtain Kubernetes config outside of cluster: " + clientgocmdtools.RecommendedHomeFile)
|
||||
> config, err = clientgorest.InClusterConfig()
|
||||
> if err != nil {
|
||||
> log.Error("Failed to obtain Kubernetes config both inside/outside of cluster, the DevContainer is Disabled")
|
||||
> setting.Devstar.Devcontainer.Enabled = false
|
||||
> return nil, err
|
||||
> }
|
||||
> }
|
||||
>
|
||||
> // 2. 根据 k8s 配置信息构建 ClientSet
|
||||
> dynamicClient, err := dynamicclient.NewForConfig(config)
|
||||
> if err != nil {
|
||||
> return nil, err
|
||||
> }
|
||||
> return dynamicClient, err
|
||||
> }
|
||||
>
|
||||
> ```
|
||||
>
|
||||
> 上述代码考虑了集群内、外两种情况配置 k8s 连接信息:
|
||||
> - 集群之外/Pod 之外:通过 `~/.kube/config` 获取 Token 与 CA 证书文件
|
||||
> - 集群 Pod 之内:通过 Pod 内部密钥挂载目录 `/var/run/secrets/kubernetes.io/serviceaccount/` 获取 Token 与 CA 证书文件
|
||||
>
|
||||
|
||||
--------------------
|
||||
## Gitea
|
||||
|
||||
616
docs/content/development/api-devstar.zh-cn.md
Normal file
616
docs/content/development/api-devstar.zh-cn.md
Normal file
@@ -0,0 +1,616 @@
|
||||
---
|
||||
date: "2018-06-24:00:00+02:00"
|
||||
title: "DevStar API 使用指南"
|
||||
slug: "api-devstar"
|
||||
sidebar_position: 40
|
||||
toc: false
|
||||
draft: false
|
||||
aliases:
|
||||
- /zh-cn/api-devstar
|
||||
menu:
|
||||
sidebar:
|
||||
parent: "development"
|
||||
name: "DevStar API 使用指南"
|
||||
sidebar_position: 40
|
||||
identifier: "api-devstar"
|
||||
---
|
||||
|
||||
|
||||
# DevStar API 使用指南
|
||||
|
||||
## 1. 生成临时 SSH 密钥对
|
||||
|
||||
**请求方式**:
|
||||
|
||||
```curl
|
||||
GET /api/devstar_ssh/key_pair/new_temp?_=${CurrentTimestamp}
|
||||
```
|
||||
|
||||
| 请求参数 | 类型 | 含义 | 作用 |
|
||||
| ------- | ---- | ----------------------------------- | ---- |
|
||||
| `_` | 整数 | 匿名参数,存放任意值(推荐使用当前的时间戳)| 用于防止 HTTP GET 请求被缓存,保证每次请求都能到达服务器端|
|
||||
|
||||
|
||||
| 请求头 | 载荷 | 作用 |
|
||||
| -------------- | --------------------- | --- |
|
||||
| `Authorization` | `token ${accessToken}`| 登录凭证,用于标识用户身份,防止接口被滥用 |
|
||||
| `Content-Type` | `application/json` | 用于标识返回数据格式是 JSON |
|
||||
|
||||
**响应格式(操作成功)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"msg": "操作成功",
|
||||
"data": {
|
||||
"KeySize": "2048",
|
||||
"privateKeyPem": "-----BEGIN RSA PRIVATE KEY-----\n......\n-----END RSA PRIVATE KEY-----\n",
|
||||
"publicKeyFingerprint": "ssh-rsa AAAAB3N......\n",
|
||||
"publicKeyPem": "-----BEGIN PUBLIC KEY-----\n......\n-----END PUBLIC KEY-----\n"
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码,`0` 表示无错误
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码的具体含义, `0` 代表操作成功
|
||||
- `data`: 返回数据
|
||||
- `KeySize`: 当前系统使用的密钥长度,默认值 2048,可在 app.ini 文件中的 `devstar.ssh_key_pair.KEY_SIZE` 指定大于 2048 位的密钥长度
|
||||
- `privateKeyPem`: PEM 格式的 RSA 私钥,可保存于客户端目录 `~/.ssh/id_rsa` 用于 SSH 登录使用的 RSA 私钥
|
||||
- `publicKeyPem`: PEM 格式的 RSA 公钥
|
||||
- `publicKeyFingerprint`: SSH 格式的 RSA 公钥,可保存于服务器目录 `~/.ssh/authorized_keys` 用于 SSH 登录使用的 RSA 公钥
|
||||
|
||||
|
||||
**响应格式(操作失败:未登录)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 1,
|
||||
"msg": "未登录,禁止访问"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码,`1` 表示发生错误
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码 `1` 具体含义: 未登录,禁止访问
|
||||
|
||||
|
||||
**响应格式(操作失败:发生内部错误)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 12001,
|
||||
"msg": "生成 SSH 密钥对失败",
|
||||
"data": {
|
||||
"ErrorMsg": "Failed to ${ACTION_NAME} for SSH Key: ${ACTION_RESULT}",
|
||||
"KeySize": "2048"
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
- `data`: 对错误信息进一步描述
|
||||
- `KeySize`: 当前系统使用的密钥长度,默认值 2048,可在 app.ini 文件中的 `devstar.ssh_key_pair.KEY_SIZE` 指定大于 2048 位的密钥长度
|
||||
- `ErrorMsg`: API 调用过程中返回错误信息封装,其中 `ACTION_NAME` 表示具体操作步骤,`ACTION_RESULT` 表示具体操作步骤结果
|
||||
|
||||
|
||||
## 2. DevContainer 管理
|
||||
|
||||
DevContainer 是指可以通过 SSH 连接的远程服务器开发容器环境
|
||||
|
||||
|
||||
> 在 k8s 中,DevContainer 被定义为 CRD 资源 `DevcontainerApp`,具体包含如下资源:
|
||||
> - `StatefulSet`: 用于管理有状态应用
|
||||
> - `Service`:用于向集群外暴露 DevContainer,提供服务访问
|
||||
>
|
||||
>
|
||||
> k8s CRD 资源 `DevcontainerApp` 通过 k8s Operator 机制进行管理,可以通过下列 kubectl 命令查看:
|
||||
>
|
||||
> ```bash
|
||||
> kubectl get deployment -n devcontainer-operator-system devcontainer-operator-controller-manager
|
||||
> # NAME READY UP-TO-DATE AVAILABLE AGE
|
||||
> # devcontainer-operator-controller-manager 1/1 1 1 2d17h
|
||||
>
|
||||
> ```
|
||||
>
|
||||
> 对于部署在 k8s 上的 DevContainer,SSH 连接依此经过:
|
||||
> 1. 云服务厂商的 NAT 路由器
|
||||
> 2. k8s NodePort Service
|
||||
> 3. k8s Pod
|
||||
> 4. k8s Pod 中运行的 OpenSSH 服务器
|
||||
>
|
||||
|
||||
|
||||
对 DevContainer 管理主要包括下述操作:
|
||||
- 创建一个新的 DevContainer
|
||||
- 查看当前用户创建的所有 DevContainer
|
||||
- 获取 DevContainer 打开信息
|
||||
- 删除 DevContainer
|
||||
|
||||
|
||||
|
||||
### 2.1 创建 DevContainer
|
||||
|
||||
|
||||
**请求方式**:
|
||||
|
||||
```curl
|
||||
POST /api/devcontainer
|
||||
```
|
||||
|
||||
|
||||
| 请求参数 | 类型 | 含义 | 作用 |
|
||||
| ----------------- | -------- | ---------- | ---- |
|
||||
| `repoId` | 字符串 | 仓库 ID | 标识当前用户希望创建 DevContainer 所关联的 仓库 |
|
||||
| `sshPublicKeyList`| 字符串数组 | SSH公钥列表 | 标识用户除了使用 DevStar 后台添加的用户永久 SSH 公钥外,该 DevContainer SSH 会话使用额外的临时 SSH 公钥列表|
|
||||
|
||||
注:上述请求参数需要以 JSON 格式封装在 POST 请求 Body 域:
|
||||
|
||||
```json
|
||||
{
|
||||
"repoId": "${REPO_ID}",
|
||||
"sshPublicKeyList": [
|
||||
"ssh-rsa AAAAB3N......Pn9TWOE= SSH_PUBLIC_KEY_1",
|
||||
"ssh-rsa AAAAB3N......Pn9TWOE= SSH_PUBLIC_KEY_2",
|
||||
]
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
|
||||
| 请求头 | 载荷 | 作用 |
|
||||
| -------------- | --------------------- | --- |
|
||||
| `Authorization` | `token ${accessToken}`| 登录凭证,用于标识用户身份,防止接口被滥用 |
|
||||
| `Content-Type` | `application/json` | 用于标识返回数据格式是 JSON |
|
||||
|
||||
**响应格式(操作成功)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"msg": "操作成功"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码,`0` 表示无错误
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码的具体含义, `0` 代表创建 DevContainer 成功
|
||||
|
||||
|
||||
**响应格式(操作失败:未登录)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 1,
|
||||
"msg": "未登录,禁止访问"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
|
||||
|
||||
**响应格式(操作失败:数据校验失败)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 11002,
|
||||
"msg": "无效参数",
|
||||
"data": {
|
||||
"ErrorMsg": "......"
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
- `data`: 对错误信息进一步描述
|
||||
- `ErrorMsg`: 描述数据校验失败的报错信息(可能是传入 `repoId` 不是数字,或者 `repoId` 字符串长度过长)
|
||||
|
||||
|
||||
|
||||
**响应格式(操作失败:发生内部错误)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 11003,
|
||||
"msg": "创建 DevContainer 失败",
|
||||
"data": {
|
||||
"ErrorMsg": "Failed to ${ACTION_NAME} in DevContainer Service: ${ACTION_RESULT}"
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
- `data`: 对错误信息进一步描述
|
||||
- `ErrorMsg`: 描述数据校验失败的报错信息,`${ACTION_NAME}` 表示操作名称,`${ACTION_RESULT}` 表示操作结果
|
||||
|
||||
|
||||
### 2.2 获取当前登录用户的 DevContainer 列表
|
||||
|
||||
|
||||
**请求方式**:
|
||||
|
||||
```curl
|
||||
GET /api/devcontainer/user?_=${CurrentTimestamp}&page=1&page_size=2
|
||||
```
|
||||
|
||||
| 请求参数 | 类型 | 含义 | 作用 |
|
||||
| ----------- | ---- | --------------------------------------- | ---- |
|
||||
| `_` | 整数 | 匿名参数,存放任意值(推荐使用当前的时间戳)| 用于防止 HTTP GET 请求被缓存,保证每次请求都能到达服务器端|
|
||||
| `page` | 整数 | 当前所在页码(默认值 `1`) | 用于分页展示 DevContainer |
|
||||
| `page_size` | 整数 | 当前所在页码(必须小于等于默认值:`app.ini` 中的 `ui.admin.DEV_CONTAINERS_PAGING_NUM`) | 用于控制每页展示 DevContainer 个数|
|
||||
|
||||
|
||||
| 请求头 | 载荷 | 作用 |
|
||||
| -------------- | --------------------- | --- |
|
||||
| `Authorization` | `token ${accessToken}`| 登录凭证,用于标识用户身份,防止接口被滥用 |
|
||||
| `Content-Type` | `application/json` | 用于标识返回数据格式是 JSON |
|
||||
|
||||
**响应格式(操作成功)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"msg": "操作成功",
|
||||
"data": {
|
||||
"userId": 5,
|
||||
"username": "daimingchen",
|
||||
"devContainers": [
|
||||
{
|
||||
"devContainerId": 41,
|
||||
"devContainerName": "daimingchen-devstar-fa72bebd8bb611ef9c1a4e1bce2a7080",
|
||||
"devContainerHost": "devcontainer.devstar.cn",
|
||||
"devContainerUsername": "root",
|
||||
"devContainerWorkDir": "/data",
|
||||
"repoId": 3,
|
||||
"repoName": "devstar",
|
||||
"repo_owner_name": "devstar",
|
||||
"repo_link": "/devstar/devstar",
|
||||
"repoDescription": "DevStar Studio"
|
||||
},
|
||||
{
|
||||
"devContainerId": 38,
|
||||
"devContainerName": "daimingchen-12-dcbb9c5b8a0611ef9c1a4e1bce2a7080",
|
||||
"devContainerHost": "devcontainer.devstar.cn",
|
||||
"devContainerUsername": "root",
|
||||
"devContainerWorkDir": "/data",
|
||||
"repoId": 19,
|
||||
"repoName": "12",
|
||||
"repo_owner_name": "leviyanx",
|
||||
"repo_link": "/leviyanx/12"
|
||||
}
|
||||
],
|
||||
"page": 1,
|
||||
"pageSize": 30,
|
||||
"pageTotalNum": 1,
|
||||
"itemTotalNum": 2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码,`0` 表示无错误
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码的具体含义, `0` 代表获取用户 DevContainer 列表成功
|
||||
- `data`: 返回数据
|
||||
- `userId`: 当前已登录用户 ID
|
||||
- `username`: 当前已登录用户名
|
||||
- `itemTotalNum`: 当前用户 DevContainer 总数
|
||||
- `pageSize`: 每个页面最多展示 DevContainer 个数
|
||||
- `page`: 当前页码
|
||||
- `devContainers`: 当前页面 DevContainer 列表
|
||||
- `pageTotalNum`: 总页数
|
||||
|
||||
其中,对于 `devContainers` 中的每个元素,具体含义如下:
|
||||
|
||||
|字段名称|类型|含义|
|
||||
|--|--|--|
|
||||
|`devContainerId`| 整数| DevContainer ID |
|
||||
|`devContainerName`|字符串| 唯一标识 DevContainer 名称 |
|
||||
|`devContainerHost`|字符串| SSH 连接主机的 IP 或者 DNS 域名 |
|
||||
|`devContainerUsername`|字符串| SSH 登录用户名 |
|
||||
|`devContainerWorkDir`|字符串| SSH 登录成功后工作目录 |
|
||||
|`repoId`|整数| DevContainer 关联的仓库的 ID |
|
||||
|`repoName`|字符串| DevContainer 关联的仓库的名称 |
|
||||
|`repo_owner_name`|字符串| DevContainer 关联的仓库的所有者用户 ID |
|
||||
|`repo_link`|字符串| DevContainer 关联的仓库的访问绝对地址 URL |
|
||||
|`repoDescription`|字符串| DevContainer 关联的仓库的描述信息(可选字段) |
|
||||
|
||||
<!--
|
||||
|
||||
TODO: 检查所有返回前端的数据,将 int64 类型替换为字符串
|
||||
- devContainerId
|
||||
- repoId
|
||||
-->
|
||||
|
||||
**响应格式(操作失败:未登录)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 1,
|
||||
"msg": "未登录,禁止访问"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
|
||||
|
||||
**响应格式(操作失败:发生内部错误)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 11006,
|
||||
"msg": "查询用户 DevContainer 列表失败",
|
||||
"data": {
|
||||
"ErrorMsg": "Failed to ${ACTION_NAME} in DevStar DevContainer DB: ${ACTION_RESULT}"
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
- `data`: 对错误信息进一步描述
|
||||
- `ErrorMsg`: 描述数据校验失败的报错信息,`${ACTION_NAME}` 表示操作名称,`${ACTION_RESULT}` 表示操作结果
|
||||
|
||||
|
||||
|
||||
### 2.3 打开 DevContainer
|
||||
|
||||
**请求方式**:
|
||||
|
||||
```curl
|
||||
GET /api/devcontainer?_=${CurrentTimestamp}&repoId=${REPO_ID}&wait=${IS_WAITING}
|
||||
```
|
||||
|
||||
|
||||
| 请求参数 | 类型 | 含义 | 作用 |
|
||||
| -------- | ---- | --------------------------------------- | ---- |
|
||||
| `_` | 整数 | 匿名参数,存放任意值(推荐使用当前的时间戳)| 用于防止 HTTP GET 请求被缓存,保证每次请求都能到达服务器端|
|
||||
| `repoId` | 字符串 | 仓库 ID | 根据仓库打开关联的 DevContainer|
|
||||
| `wait` | 布尔 | 是否阻塞等待 DevContainer 就绪 | 是否注册监听器,在超时时间内阻塞等待 DevContainer 就绪 |
|
||||
|
||||
|
||||
| 请求头 | 载荷 | 作用 |
|
||||
| -------------- | --------------------- | --- |
|
||||
| `Authorization` | `token ${accessToken}`| 登录凭证,用于标识用户身份,防止接口被滥用 |
|
||||
| `Content-Type` | `application/json` | 用于标识返回数据格式是 JSON |
|
||||
|
||||
**响应格式(操作成功)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"msg": "操作成功",
|
||||
"data": {
|
||||
"devContainerId": 38,
|
||||
"devContainerName": "daimingchen-12-dcbb9c5b8a0611ef9c1a4e1bce2a7080",
|
||||
"devContainerHost": "devcontainer.devstar.cn",
|
||||
"devContainerUsername": "root",
|
||||
"devContainerWorkDir": "/data",
|
||||
"repoId": 19,
|
||||
"repoName": "12",
|
||||
"repo_owner_name": "leviyanx",
|
||||
"repo_link": "/leviyanx/12",
|
||||
"devContainerPort": 30000
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码,`0` 表示无错误
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码的具体含义, `0` 代表获取用户 DevContainer 连接信息成功
|
||||
- `data`: 返回数据
|
||||
|
||||
|`data` 字段名称|类型|含义|
|
||||
|--|--|--|
|
||||
|`devContainerId`| 整数| DevContainer ID |
|
||||
|`devContainerName`|字符串| 唯一标识 DevContainer 名称 |
|
||||
|`devContainerHost`|字符串| SSH 连接主机的 IP 或者 DNS 域名 |
|
||||
|`devContainerPort`|整数|SSH 连接主机的端口号|
|
||||
|`devContainerUsername`|字符串| SSH 登录用户名 |
|
||||
|`devContainerWorkDir`|字符串| SSH 登录成功后工作目录 |
|
||||
|`repoId`|整数| DevContainer 关联的仓库的 ID |
|
||||
|`repoName`|字符串| DevContainer 关联的仓库的名称 |
|
||||
|`repo_owner_name`|字符串| DevContainer 关联的仓库的所有者用户 ID |
|
||||
|`repo_link`|字符串| DevContainer 关联的仓库的访问绝对地址 URL |
|
||||
|`repoDescription`|字符串| DevContainer 关联的仓库的描述信息(可选字段) |
|
||||
|
||||
|
||||
**响应格式(操作失败:未登录)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 1,
|
||||
"msg": "未登录,禁止访问"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
|
||||
|
||||
**响应格式(操作失败:无效参数)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 11002,
|
||||
"msg": "无效参数"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
|
||||
|
||||
**响应格式(操作失败:发生内部错误,包括 DevContainer 未在超时时间内就绪)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 11004,
|
||||
"msg": "打开 DevContainer 失败",
|
||||
"data": {
|
||||
"ErrorMsg": "......"
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
响应数据说明:
|
||||
- `code`: 错误代码
|
||||
- `msg`: 错误代码含义,以人类可读的方式描述错误代码具体含义
|
||||
- `data`: 对错误信息进一步描述
|
||||
- `ErrorMsg`: 描述数据校验失败的报错信息
|
||||
- 参数无效:Illegal Params
|
||||
- DevContainer 未找到:DevContainer NOT found in repo '`${REPO_NAME}`'(repoId = `${REPO_ID}`) of user '`${USERNAME}`'(userId = `${USER_ID}`)
|
||||
- 其他:Failed to `${ACTION_NAME}` in DevContainer Service: `${ACTION_RESULT}`
|
||||
|
||||
|
||||
> **补充**: 如何判断 DevContainer 就绪状态
|
||||
>
|
||||
> 在 k8s 中,DevContainer 以 k8s CRD `DevcontainerApp` 资源定义,可参考如下 YAML:
|
||||
>
|
||||
> ```yaml
|
||||
> apiVersion: devcontainer.devstar.cn/v1
|
||||
> kind: DevcontainerApp
|
||||
> metadata:
|
||||
> name: studio-test
|
||||
> namespace: devstar-studio-ns
|
||||
> spec:
|
||||
> statefulset:
|
||||
> image: devstar.cn/public/base-ssh-devcontainer:ubuntu-20.04-20241014
|
||||
> gitRepositoryURL: https://gitee.com/daimingchen_gitee/mock-repo
|
||||
> command:
|
||||
> - /bin/bash
|
||||
> - -c
|
||||
> - service ssh restart && tail -f /dev/null
|
||||
> containerPort: 22
|
||||
> sshPublicKeyList:
|
||||
> - ssh-rsa AAA......e8 SSHPublicKey
|
||||
> - ssh-rsa AAA......e9 YetAnotherSSHPublicKey
|
||||
>
|
||||
> ```
|
||||
>
|
||||
> 使用 `kubectl` 创建 `DevcontainerApp`:
|
||||
>
|
||||
> ```bash
|
||||
> kubectl apply -f devcontainerapp-demo.yaml
|
||||
> ```
|
||||
>
|
||||
> 等待一段时间后,获取 DevcontainerApp `studio-test` 的状态:
|
||||
>
|
||||
> ```bash
|
||||
> kubectl get devcontainerapp -n devstar-studio-ns studio-test -o yaml
|
||||
> ```
|
||||
>
|
||||
> 即可得到下列 YAML
|
||||
>
|
||||
> ```yaml
|
||||
> apiVersion: devcontainer.devstar.cn/v1
|
||||
> kind: DevcontainerApp
|
||||
> metadata:
|
||||
> name: studio-test
|
||||
> namespace: devstar-studio-ns
|
||||
> creationTimestamp: "2024-10-16T12:05:47Z"
|
||||
> generation: 1
|
||||
> resourceVersion: "92973809"
|
||||
> uid: 867c8bc6-1b79-4c41-8659-686b176f4c56
|
||||
> spec:
|
||||
> statefulset:
|
||||
> image: devstar.cn/public/base-ssh-devcontainer:ubuntu-20.04-20241014
|
||||
> gitRepositoryURL: https://gitee.com/daimingchen_gitee/mock-repo
|
||||
> command:
|
||||
> - /bin/bash
|
||||
> - -c
|
||||
> - service ssh restart && tail -f /dev/null
|
||||
> containerPort: 22
|
||||
> sshPublicKeyList:
|
||||
> - ssh-rsa AAA......e8 SSHPublicKey
|
||||
> - ssh-rsa AAA......e9 YetAnotherSSHPublicKey
|
||||
> status:
|
||||
> nodePortAssigned: 30000
|
||||
> ready: true
|
||||
>
|
||||
> ```
|
||||
>
|
||||
> 更详细地,`status.ready` 域具体赋值由下列两个指标判断:
|
||||
> - 分配的 NodePort 端口值 `nodePortAssigned` 取值范围在左闭右闭区间 $[30000, 32767]$
|
||||
> - StatefulSet 控制下的 Pod 中的容器就绪探针探测成功
|
||||
>
|
||||
> 总之,在 DevStar Studio 中,只需要读取集群中 k8s CRD `DevcontainerApp` 的 `status.ready` 域即可判断 DevContainer 是否就绪
|
||||
>
|
||||
|
||||
|
||||
|
||||
### 2.4 删除 DevContainer
|
||||
|
||||
**请求方式**:
|
||||
|
||||
```curl
|
||||
DELETE /api/devcontainer?repoId=${REPO_ID}
|
||||
```
|
||||
|
||||
|
||||
| 请求参数 | 类型 | 含义 | 作用 |
|
||||
| -------- | ------ | ------ | ---------------------------- |
|
||||
| `repoId` | 字符串 | 仓库 ID | 根据仓库打开关联的 DevContainer|
|
||||
|
||||
|
||||
| 请求头 | 载荷 | 作用 |
|
||||
| -------------- | --------------------- | --- |
|
||||
| `Authorization` | `token ${accessToken}`| 登录凭证,用于标识用户身份,防止接口被滥用 |
|
||||
| `Content-Type` | `application/json` | 用于标识返回数据格式是 JSON |
|
||||
|
||||
**响应格式(操作成功)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 0,
|
||||
"msg": "操作成功"
|
||||
}
|
||||
```
|
||||
|
||||
**响应格式(操作失败:未登录)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 1,
|
||||
"msg": "未登录,禁止访问"
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
|
||||
**响应格式(操作失败:未找到 DevContainer)**:
|
||||
|
||||
```json
|
||||
{
|
||||
"code": 11005,
|
||||
"msg": "删除 DevContainer 失败",
|
||||
"data": {
|
||||
"ErrorMsg": "Illegal Params: [opts.RepoId]"
|
||||
}
|
||||
}
|
||||
```
|
||||
2
go.mod
2
go.mod
@@ -100,6 +100,8 @@ require (
|
||||
github.com/shurcooL/vfsgen v0.0.0-20230704071429-0000e147ea92
|
||||
github.com/stretchr/testify v1.9.0
|
||||
github.com/syndtr/goleveldb v1.0.0
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.1027
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/vpc v1.0.1027
|
||||
github.com/tstranex/u2f v1.0.0
|
||||
github.com/ulikunitz/xz v0.5.12
|
||||
github.com/urfave/cli/v2 v2.27.2
|
||||
|
||||
4
go.sum
4
go.sum
@@ -923,6 +923,10 @@ github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8
|
||||
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
|
||||
github.com/syndtr/goleveldb v1.0.0 h1:fBdIW9lB4Iz0n9khmH8w27SJ3QEJ7+IgjPEwGSZiFdE=
|
||||
github.com/syndtr/goleveldb v1.0.0/go.mod h1:ZVVdQEZoIme9iO1Ch2Jdy24qqXrMMOU6lpPAyBWyWuQ=
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.1027 h1:4/chUqYM26idp2Rd3pPmsL94RTAwIM5dGXoaskOQ+cM=
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.0.1027/go.mod h1:r5r4xbfxSaeR04b166HGsBa/R4U3SueirEUpXGuw+Q0=
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/vpc v1.0.1027 h1:yPaYhREjAhm/RdcFQoXB4Hql4+kIESL7Vzk3wKYwvqo=
|
||||
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/vpc v1.0.1027/go.mod h1:FtdjjXaY75iqpsglFL0AshphAMOyScI41tRp1E9VgQY=
|
||||
github.com/tinylib/msgp v1.1.0/go.mod h1:+d+yLhGm8mzTaHzB+wgMYrodPfmZrzkirds8fDWklFE=
|
||||
github.com/toqueteos/webbrowser v1.2.0 h1:tVP/gpK69Fx+qMJKsLE7TD8LuGWPnEV71wBN9rrstGQ=
|
||||
github.com/toqueteos/webbrowser v1.2.0/go.mod h1:XWoZq4cyp9WeUeak7w7LXRUQf1F1ATJMir8RTqb4ayM=
|
||||
|
||||
@@ -18,6 +18,8 @@ type DevstarDevcontainer struct {
|
||||
DevcontainerWorkDir string `xorm:"VARCHAR(256) charset=utf8mb4 collate=utf8mb4_bin NOT NULL 'devcontainer_work_dir' comment('SSH 工作路径,典型值 ~/${project_name},256字节以内')"`
|
||||
RepoId int64 `xorm:"BIGINT NOT NULL 'repo_id' comment('repository表主键')"`
|
||||
UserId int64 `xorm:"BIGINT NOT NULL 'user_id' comment('user表主键')"`
|
||||
CreatedUnix int64 `xorm:"BIGINT 'created_unix' comment('创建时间戳')"`
|
||||
UpdatedUnix int64 `xorm:"BIGINT 'updated_unix' comment('更新时间戳')"`
|
||||
}
|
||||
|
||||
func init() {
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
package k8s_agent
|
||||
|
||||
import (
|
||||
"code.gitea.io/gitea/modules/setting"
|
||||
"code.gitea.io/gitea/services/devstar_cloud_provider"
|
||||
"code.gitea.io/gitea/services/devstar_devcontainer/errors"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
@@ -101,7 +104,11 @@ func CreateDevcontainer(ctx *context.Context, client dynamic_client.Interface, o
|
||||
//log.Info("DevContainer NodePort Status 更新完成,最新 NodePort = %v", nodePortAssigned)
|
||||
//break
|
||||
// 收到 NodePort Service MODIFIED 消息后,更新 NodePort,直接返回,不再处理后续 Event (否则必须超时3秒才得到 NodePort)
|
||||
return devcontainerApp, nil
|
||||
natRuleDescription := "DevContainer: " + devcontainerApp.Name
|
||||
privatePort := uint64(nodePortAssigned)
|
||||
publicPort := privatePort
|
||||
err = devstar_cloud_provider.CreateNATRulePort(privatePort, publicPort, natRuleDescription)
|
||||
return devcontainerApp, err
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -124,5 +131,19 @@ func CreateDevcontainer(ctx *context.Context, client dynamic_client.Interface, o
|
||||
// }
|
||||
//}
|
||||
*/
|
||||
return devcontainerApp, nil
|
||||
|
||||
// 如果执行到这里,说明 k8s 集群中 DevcontainerApp 初始化失败,比如执行下列命令查看出错原因如下:
|
||||
// $ kubectl get pod -n devstar-studio-ns test-mockrepo1-6c5369588f8911e-0
|
||||
// NAME READY STATUS RESTARTS AGE
|
||||
// test-mockrepo1-6c5369588f8911e-0 0/1 Init:CrashLoopBackOff 1 (5s ago) 7s
|
||||
// 需要删除刚刚创建的 k8s CRD,然后返回 DevContainer 初始化失败
|
||||
optsDeleteInitFailed := &devcontainer_dto.DeleteDevcontainerOptions{
|
||||
Namespace: setting.Devstar.Devcontainer.Namespace,
|
||||
Name: devcontainerApp.Name,
|
||||
}
|
||||
_ = DeleteDevcontainer(ctx, client, optsDeleteInitFailed)
|
||||
return nil, errors.ErrOperateDevcontainer{
|
||||
Action: "Initialize DevContainer",
|
||||
Message: fmt.Sprintf("DevContainer %v failed to initialize and is thus purged.", devcontainerApp.Name),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,9 +15,19 @@ var validDevcontainerAgentSet = map[string]struct{}{
|
||||
DEVCONTAINER_AGENT_NAME_DOCKER: {},
|
||||
}
|
||||
|
||||
const (
|
||||
CLOUD_PROVIDER_TENCENT = "tencent"
|
||||
)
|
||||
|
||||
// validCloudProviderSet 私有 Set 结构,标识目前系统所有支持的 Cloud Provider 类型
|
||||
var validCloudProviderSet = map[string]struct{}{
|
||||
CLOUD_PROVIDER_TENCENT: {},
|
||||
}
|
||||
|
||||
var Devstar = struct {
|
||||
Devcontainer DevcontainerType `ini:"devstar.devcontainer"`
|
||||
SSHKeypair SSHKeyPairType `ini:"devstar.ssh_key_pair"`
|
||||
Cloud CloudType `ini:"devstar.cloud"`
|
||||
}{
|
||||
Devcontainer: DevcontainerType{
|
||||
Enabled: false,
|
||||
@@ -27,6 +37,9 @@ var Devstar = struct {
|
||||
SSHKeypair: SSHKeyPairType{
|
||||
KeySize: 2048,
|
||||
},
|
||||
Cloud: CloudType{
|
||||
Enabled: false,
|
||||
},
|
||||
}
|
||||
|
||||
type DevcontainerType struct {
|
||||
@@ -41,6 +54,23 @@ type SSHKeyPairType struct {
|
||||
KeySize int
|
||||
}
|
||||
|
||||
type CloudType struct {
|
||||
Enabled bool
|
||||
Provider string
|
||||
Tencent CloudProviderTencentType `ini:"devstar.cloud.tencent"`
|
||||
}
|
||||
|
||||
type CloudProviderTencentType struct {
|
||||
Endpoint string
|
||||
Region string
|
||||
NatGatewayId string
|
||||
PublicIpAddress string
|
||||
PrivateIpAddress string
|
||||
IpProtocol string
|
||||
SecretId string
|
||||
SecretKey string
|
||||
}
|
||||
|
||||
// validateDevstarDevcontainerSettings 检查从 ini 配置文件中读取 DevStar DevContainer 配置信息,若数据无效,则自动禁用 DevContainer
|
||||
func validateDevstarDevcontainerSettings() {
|
||||
|
||||
@@ -70,9 +100,57 @@ func validateDevstarSSHKeyPairSettings() {
|
||||
}
|
||||
}
|
||||
|
||||
// validateDevstarCloudSettings 检查从 ini 配置文件中读取 DevStar Cloud 配置信息
|
||||
func validateDevstarCloudSettings() {
|
||||
switch Devstar.Cloud.Provider {
|
||||
case CLOUD_PROVIDER_TENCENT:
|
||||
// 腾讯云配置检查
|
||||
|
||||
if len(Devstar.Cloud.Tencent.NatGatewayId) < 4 {
|
||||
log.Warn("INVALID NAT Gateway ID '%v' for DevStar Cloud Provider Tencent", Devstar.Cloud.Tencent.NatGatewayId)
|
||||
Devstar.Cloud.Enabled = false
|
||||
}
|
||||
|
||||
if Devstar.Cloud.Tencent.IpProtocol != "TCP" && Devstar.Cloud.Tencent.IpProtocol != "UDP" {
|
||||
log.Warn("INVALID IP Protocol '%v' for DevStar Cloud Provider Tencent", Devstar.Cloud.Tencent.IpProtocol)
|
||||
Devstar.Cloud.Enabled = false
|
||||
}
|
||||
|
||||
if len(Devstar.Cloud.Tencent.Region) < 3 || len(Devstar.Cloud.Tencent.Endpoint) == 0 {
|
||||
log.Warn("INVALID (Region, Endpoint) pair ('%v', '%v') for DevStar Cloud Provider Tencent",
|
||||
Devstar.Cloud.Tencent.Region, Devstar.Cloud.Tencent.Endpoint)
|
||||
Devstar.Cloud.Enabled = false
|
||||
}
|
||||
|
||||
if len(Devstar.Cloud.Tencent.PrivateIpAddress) == 0 || len(Devstar.Cloud.Tencent.PublicIpAddress) == 0 {
|
||||
log.Warn("INVALID (PublicIpAddress, PrivateIpAddress) pair ('%v', '%v') for DevStar Cloud Provider Tencent",
|
||||
Devstar.Cloud.Tencent.PublicIpAddress, Devstar.Cloud.Tencent.PrivateIpAddress)
|
||||
Devstar.Cloud.Enabled = false
|
||||
}
|
||||
|
||||
if len(Devstar.Cloud.Tencent.SecretId) == 0 || len(Devstar.Cloud.Tencent.SecretKey) == 0 {
|
||||
log.Warn("INVALID (SecretId, SecretKey) pair for DevStar Cloud Provider Tencent")
|
||||
Devstar.Cloud.Enabled = false
|
||||
}
|
||||
|
||||
default:
|
||||
// 无效 Cloud Provider 名称
|
||||
log.Warn("INVALID config '%v' for DevStar Cloud", Devstar.Cloud.Provider)
|
||||
Devstar.Cloud.Enabled = false
|
||||
}
|
||||
|
||||
if Devstar.Cloud.Enabled == false {
|
||||
log.Warn("DevStar Cloud Provider Service Disabled")
|
||||
} else {
|
||||
log.Info("DevStar Cloud Provider '%v' Enabled", Devstar.Cloud.Provider)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
func loadDevstarFrom(rootCfg ConfigProvider) {
|
||||
mustMapSetting(rootCfg, "devstar", &Devstar)
|
||||
|
||||
validateDevstarDevcontainerSettings()
|
||||
validateDevstarSSHKeyPairSettings()
|
||||
validateDevstarCloudSettings()
|
||||
}
|
||||
|
||||
@@ -39,7 +39,19 @@ func ListUserDevcontainers(ctx *gitea_web_context.Context) {
|
||||
PageSize: userPageSize,
|
||||
},
|
||||
}
|
||||
userDevcontainersVO, _ := devstar_devcontainer_service.GetUserDevcontainersList(ctx, opts)
|
||||
userDevcontainersVO, err := devstar_devcontainer_service.GetUserDevcontainersList(ctx, opts)
|
||||
|
||||
if err != nil {
|
||||
resultFailed2ListUserDevcontainerList := Result.ResultType{
|
||||
Code: Result.RespFailedListUserDevcontainers.Code,
|
||||
Msg: Result.RespFailedListUserDevcontainers.Msg,
|
||||
Data: map[string]string{
|
||||
"ErrorMsg": err.Error(),
|
||||
},
|
||||
}
|
||||
resultFailed2ListUserDevcontainerList.RespondJson2HttpResponseWriter(ctx.Resp)
|
||||
return
|
||||
}
|
||||
|
||||
// 3. 封装VO
|
||||
resultListUserDevcontainersVO := Result.ResultType{
|
||||
|
||||
@@ -25,3 +25,9 @@ var RespFailedDeleteDevcontainer = ResultType{
|
||||
Code: 11005,
|
||||
Msg: "删除 DevContainer 失败",
|
||||
}
|
||||
|
||||
// RespFailedListUserDevcontainers 查询用户 DevContainer 列表失败
|
||||
var RespFailedListUserDevcontainers = ResultType{
|
||||
Code: 11006,
|
||||
Msg: "查询用户 DevContainer 列表失败",
|
||||
}
|
||||
|
||||
24
services/devstar_cloud_provider/CreateNATRulePort.go
Normal file
24
services/devstar_cloud_provider/CreateNATRulePort.go
Normal file
@@ -0,0 +1,24 @@
|
||||
package devstar_cloud_provider
|
||||
|
||||
import (
|
||||
"code.gitea.io/gitea/modules/setting"
|
||||
devstar_cloud_provider_errors "code.gitea.io/gitea/services/devstar_cloud_provider/errors"
|
||||
devstar_cloud_provider_tencent_nat "code.gitea.io/gitea/services/devstar_cloud_provider/tencent/nat_port_mapping"
|
||||
)
|
||||
|
||||
// CreateNATRulePort 抽象接口,创建 NAT 端口映射规则
|
||||
func CreateNATRulePort(privatePort, publicPort uint64, description string) error {
|
||||
|
||||
if setting.Devstar.Cloud.Enabled == false {
|
||||
return devstar_cloud_provider_errors.ErrCloudNATProviderDisabled{}
|
||||
}
|
||||
|
||||
// 根据配置文件指定云服务厂商创建 NAT Rule
|
||||
switch setting.Devstar.Cloud.Provider {
|
||||
case setting.CLOUD_PROVIDER_TENCENT:
|
||||
// 指定腾讯云执行 NAT 端口创建
|
||||
return devstar_cloud_provider_tencent_nat.AssignDevstarCloudNATPortForwarding2TencentCloud(privatePort, publicPort, description)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
package errors
|
||||
|
||||
type ErrCloudNATProviderDisabled struct {
|
||||
}
|
||||
|
||||
func (err ErrCloudNATProviderDisabled) Error() string {
|
||||
return "Failed to create NAT Rule since the DevStar Cloud Provider is Disabled"
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package errors
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// ErrFailed2CreateNATRule 封装创建 NAT 失败消息的结构化数据
|
||||
type ErrFailed2CreateNATRule struct {
|
||||
CloudProviderName string
|
||||
PortDescription string
|
||||
PrivatePort uint64
|
||||
PublicPort uint64
|
||||
Reason string
|
||||
}
|
||||
|
||||
// Error 实现 error 定义的 Error 接口,将结构化的错误消息转为文本日志信息
|
||||
func (err ErrFailed2CreateNATRule) Error() string {
|
||||
return fmt.Sprintf("Failed to create NAT Rule '%s' from private port %v to public port %v on Cloud Provider '%s': %v",
|
||||
err.PortDescription, err.PrivatePort, err.PublicPort, err.CloudProviderName, err.Reason)
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
package nat_port_mapping
|
||||
|
||||
import (
|
||||
"code.gitea.io/gitea/modules/log"
|
||||
"code.gitea.io/gitea/modules/setting"
|
||||
devstar_cloud_provider_service_errors "code.gitea.io/gitea/services/devstar_cloud_provider/errors"
|
||||
"github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common"
|
||||
"github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common/profile"
|
||||
vpc "github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/vpc/v20170312"
|
||||
)
|
||||
|
||||
// AssignDevstarCloudNATPortForwarding2TencentCloud 将增加 NAT 端口映射规则的任务派发到 Tencent Client
|
||||
// API Doc: https://cloud.tencent.com/document/api/215/36720
|
||||
func AssignDevstarCloudNATPortForwarding2TencentCloud(privatePort, publicPort uint64, portDescription string) error {
|
||||
// 实例化一个认证对象,入参需要传入腾讯云账户 SecretId 和 SecretKey,此处还需注意密钥对的保密
|
||||
// 代码泄露可能会导致 SecretId 和 SecretKey 泄露,并威胁账号下所有资源的安全性。
|
||||
// 以下代码示例仅供参考,建议采用更安全的方式来使用密钥,请参见:https://cloud.tencent.com/document/product/1278/85305
|
||||
// 密钥可前往官网控制台 https://console.cloud.tencent.com/cam/capi 进行获取
|
||||
credential := common.NewCredential(
|
||||
setting.Devstar.Cloud.Tencent.SecretId,
|
||||
setting.Devstar.Cloud.Tencent.SecretKey,
|
||||
)
|
||||
|
||||
// 实例化一个client选项,可选的,没有特殊需求可以跳过
|
||||
cpf := profile.NewClientProfile()
|
||||
cpf.HttpProfile.Endpoint = setting.Devstar.Cloud.Tencent.Endpoint
|
||||
// 实例化要请求产品的client对象,clientProfile是可选的
|
||||
client, _ := vpc.NewClient(credential, setting.Devstar.Cloud.Tencent.Region, cpf)
|
||||
|
||||
// 实例化一个请求对象,每个接口都会对应一个request对象
|
||||
request := vpc.NewCreateNatGatewayDestinationIpPortTranslationNatRuleRequest()
|
||||
request.NatGatewayId = common.StringPtr(setting.Devstar.Cloud.Tencent.NatGatewayId)
|
||||
request.DestinationIpPortTranslationNatRules = []*vpc.DestinationIpPortTranslationNatRule{
|
||||
&vpc.DestinationIpPortTranslationNatRule{
|
||||
IpProtocol: common.StringPtr(setting.Devstar.Cloud.Tencent.IpProtocol),
|
||||
PublicIpAddress: common.StringPtr(setting.Devstar.Cloud.Tencent.PublicIpAddress),
|
||||
PublicPort: common.Uint64Ptr(publicPort),
|
||||
PrivateIpAddress: common.StringPtr(setting.Devstar.Cloud.Tencent.PrivateIpAddress),
|
||||
PrivatePort: common.Uint64Ptr(privatePort),
|
||||
Description: common.StringPtr(portDescription),
|
||||
},
|
||||
}
|
||||
|
||||
// 返回的resp是一个CreateNatGatewayDestinationIpPortTranslationNatRuleResponse的实例,与请求对象对应
|
||||
response, err := client.CreateNatGatewayDestinationIpPortTranslationNatRule(request)
|
||||
if err != nil {
|
||||
return devstar_cloud_provider_service_errors.ErrFailed2CreateNATRule{
|
||||
CloudProviderName: "Tencent",
|
||||
PortDescription: portDescription,
|
||||
PrivatePort: privatePort,
|
||||
PublicPort: publicPort,
|
||||
Reason: err.Error(),
|
||||
}
|
||||
}
|
||||
|
||||
// 创建成功,打印留存 RequestId 同时返回 error=nil
|
||||
log.Info("Successfully created NAT Rule '%v': private port %v -> public port %v (Request ID: %v)",
|
||||
portDescription, privatePort, publicPort, response.Response.RequestId,
|
||||
)
|
||||
return nil
|
||||
}
|
||||
@@ -7,12 +7,14 @@ import (
|
||||
"code.gitea.io/gitea/modules/setting"
|
||||
DevcontainersVO "code.gitea.io/gitea/routers/api/devcontainer/vo"
|
||||
devcontainer_service_dto "code.gitea.io/gitea/services/devstar_devcontainer/dto"
|
||||
devcontainer_service_errors "code.gitea.io/gitea/services/devstar_devcontainer/errors"
|
||||
devcontainer_k8s_agent_service "code.gitea.io/gitea/services/devstar_devcontainer/k8s_agent"
|
||||
"context"
|
||||
"fmt"
|
||||
"github.com/google/uuid"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
"xorm.io/builder"
|
||||
)
|
||||
|
||||
@@ -90,22 +92,52 @@ func CreateRepoDevcontainer(ctx context.Context, opts *DevcontainersVO.CreateRep
|
||||
username := opts.Actor.Name
|
||||
repoName := opts.Repository.Name
|
||||
|
||||
// unixTimestamp is the number of seconds elapsed since January 1, 1970 UTC.
|
||||
unixTimestamp := time.Now().Unix()
|
||||
|
||||
newDevcontainer := &devcontainer_service_dto.CreateDevcontainerDTO{
|
||||
DevstarDevcontainer: devstar_devcontainer_models.DevstarDevcontainer{
|
||||
Name: getSanitizedDevcontainerName(username, repoName),
|
||||
DevcontainerHost: setting.Devstar.Devcontainer.Host,
|
||||
DevcontainerUsername: "root",
|
||||
DevcontainerWorkDir: "/data",
|
||||
DevcontainerWorkDir: "/data/workspace",
|
||||
RepoId: opts.Repository.ID,
|
||||
UserId: opts.Actor.ID,
|
||||
CreatedUnix: unixTimestamp,
|
||||
UpdatedUnix: unixTimestamp,
|
||||
},
|
||||
SSHPublicKeyList: opts.SSHPublicKeyList,
|
||||
GitRepositoryURL: strings.TrimSuffix(setting.AppURL, "/") + opts.Repository.Link(),
|
||||
}
|
||||
|
||||
// 在数据库事务中创建 Dev Container 分配资源,出错时自动回滚相对应数据库字段,保证数据一致
|
||||
dbTransactionErr := db.WithTx(ctx, func(ctx context.Context) error {
|
||||
var err error
|
||||
// 0. 查询数据库,收集用户 SSH 公钥,合并用户临时填入SSH公钥
|
||||
// (若用户合计 SSH公钥个数为0,拒绝创建DevContainer)
|
||||
/**
|
||||
SELECT content FROM public_key where owner_id = #{opts.Actor.ID}
|
||||
*/
|
||||
var userSSHPublicKeyList []string
|
||||
err = db.GetEngine(ctx).
|
||||
Table("public_key").
|
||||
Select("content").
|
||||
Where("owner_id = ?", opts.Actor.ID).
|
||||
Find(&userSSHPublicKeyList)
|
||||
if err != nil {
|
||||
return devcontainer_service_errors.ErrOperateDevcontainer{
|
||||
Action: fmt.Sprintf("query SSH Public Key List for User %s", opts.Actor.Name),
|
||||
Message: err.Error(),
|
||||
}
|
||||
}
|
||||
newDevcontainer.SSHPublicKeyList = append(userSSHPublicKeyList, opts.SSHPublicKeyList...)
|
||||
if len(userSSHPublicKeyList) <= 0 {
|
||||
// API没提供临时SSH公钥,用户后台也没有永久SSH公钥,直接结束并回滚事务
|
||||
return devcontainer_service_errors.ErrOperateDevcontainer{
|
||||
Action: "Check SSH Public Key List",
|
||||
Message: "禁止创建无法连通的DevContainer:用户未提供 SSH 公钥,请先使用API临时创建SSH密钥对、或在Web端手动添加SSH公钥",
|
||||
}
|
||||
}
|
||||
|
||||
// 1. 调用 k8s Operator Agent,创建 DevContainer 资源,同时更新k8s调度器分配的 NodePort
|
||||
err = claimDevcontainerResource(&ctx, newDevcontainer)
|
||||
if err != nil {
|
||||
@@ -218,12 +250,12 @@ func getSanitizedDevcontainerName(username, repoName string) string {
|
||||
if len(sanitizedUsername) > 15 {
|
||||
sanitizedUsername = strings.ToLower(sanitizedUsername[:15])
|
||||
}
|
||||
if len(sanitizedRepoName) > 15 {
|
||||
sanitizedRepoName = strings.ToLower(sanitizedRepoName[:15])
|
||||
if len(sanitizedRepoName) > 31 {
|
||||
sanitizedRepoName = strings.ToLower(sanitizedRepoName[:31])
|
||||
}
|
||||
newUUID, _ := uuid.NewUUID()
|
||||
uuidStr := newUUID.String()
|
||||
uuidStr = regexpNonAlphaNum.ReplaceAllString(uuidStr, "")
|
||||
uuidStr = regexpNonAlphaNum.ReplaceAllString(uuidStr, "")[:15]
|
||||
return fmt.Sprintf("%s-%s-%s", sanitizedUsername, sanitizedRepoName, uuidStr)
|
||||
}
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
devcontainer_service_errors "code.gitea.io/gitea/services/devstar_devcontainer/errors"
|
||||
devcontainer_service_options "code.gitea.io/gitea/services/devstar_devcontainer/options"
|
||||
"context"
|
||||
"fmt"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
// CreateDevcontainerAPIService API专用创建 DevContainer Service
|
||||
@@ -21,6 +21,31 @@ func CreateDevcontainerAPIService(ctx *gitea_web_context.Context, opts *devconta
|
||||
}
|
||||
}
|
||||
|
||||
// sanitize user-input SSH Public Key List
|
||||
regexpInvalidSSHPublicKey := regexp.MustCompile(`[\r]`)
|
||||
for _, publicKey := range opts.SSHPublicKeyList {
|
||||
if len(publicKey) <= 4 || publicKey[0:4] != "ssh-" || regexpInvalidSSHPublicKey.MatchString(publicKey) {
|
||||
// 遇到可能无效的 SSH Public Key,或者导致 k8s Operator YAML 解码失败的字符: `\r`,报错返回
|
||||
// ERROR Reconciler error:
|
||||
// {
|
||||
// "controller": "devcontainerapp",
|
||||
// "controllerGroup": "devcontainer.devstar.cn",
|
||||
// "controllerKind": "DevcontainerApp",
|
||||
// "DevcontainerApp": {
|
||||
// "name": "leviyanx-16-4834a4c88c4511ef9c1a4e1bce2a7080",
|
||||
// "namespace": "devstar-studio-ns"
|
||||
// },
|
||||
// "namespace": "devstar-studio-ns",
|
||||
// "name": "leviyanx-16-4834a4c88c4511ef9c1a4e1bce2a7080",
|
||||
// "reconcileID": "6af51347-7aae-4542-a5cf-2f9b57a202e6",
|
||||
// "error": "panic: error converting YAML to JSON: yaml: line 46: could not find expected ':' [recovered]"
|
||||
// }
|
||||
return devcontainer_service_errors.ErrIllegalParams{
|
||||
FieldNameList: []string{"SSHPublicKeyList"},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1. 开启事务
|
||||
errTxn := db.WithTx(ctx, func(ctx context.Context) error {
|
||||
|
||||
@@ -45,36 +70,11 @@ func CreateDevcontainerAPIService(ctx *gitea_web_context.Context, opts *devconta
|
||||
}
|
||||
}
|
||||
|
||||
// 1.3 查询数据库,收集用户 SSH 公钥,合并用户临时填入SSH公钥(若用户合计 SSH公钥个数为0,拒绝创建DevContainer)
|
||||
/**
|
||||
SELECT content FROM public_key where owner_id = #{opts.Actor.ID}
|
||||
*/
|
||||
var userSSHPublicKeyList []string
|
||||
err = db.GetEngine(ctx).
|
||||
Table("public_key").
|
||||
Select("content").
|
||||
Where("owner_id = ?", opts.Actor.ID).
|
||||
Find(&userSSHPublicKeyList)
|
||||
if err != nil {
|
||||
return devcontainer_service_errors.ErrOperateDevcontainer{
|
||||
Action: fmt.Sprintf("query SSH Public Key List for User %s", opts.Actor.Name),
|
||||
Message: err.Error(),
|
||||
}
|
||||
}
|
||||
userSSHPublicKeyList = append(userSSHPublicKeyList, opts.SSHPublicKeyList...)
|
||||
if len(userSSHPublicKeyList) <= 0 {
|
||||
// API没提供临时SSH公钥,用户后台也没有永久SSH公钥,直接结束并回滚事务
|
||||
return devcontainer_service_errors.ErrOperateDevcontainer{
|
||||
Action: "Check SSH Public Key List",
|
||||
Message: "禁止创建无法连通的DevContainer:用户未提供 SSH 公钥,请先使用API临时创建SSH密钥对、或在Web端手动添加SSH公钥",
|
||||
}
|
||||
}
|
||||
|
||||
// 1.4 调用 DevContainer Service 创建 DevContainer
|
||||
// 1.3 调用 DevContainer Service 创建 DevContainer
|
||||
optsCreateDevcontainer := &DevcontainersVO.CreateRepoDevcontainerOptions{
|
||||
Actor: opts.Actor,
|
||||
Repository: repositoryInDB,
|
||||
SSHPublicKeyList: userSSHPublicKeyList,
|
||||
SSHPublicKeyList: opts.SSHPublicKeyList,
|
||||
}
|
||||
return DevcontainersService.CreateRepoDevcontainer(ctx, optsCreateDevcontainer)
|
||||
})
|
||||
|
||||
@@ -64,7 +64,7 @@ func GenerateNewRSASSHSessionKeyPair() (error, *vo.GenerateNewRSASSHSessionKeyPa
|
||||
sshPublicKey, err := ssh.NewPublicKey(&publicKey)
|
||||
if err != nil {
|
||||
return errors.ErrGenerateNewRSASSHSessionKeyPair{
|
||||
Action: "Calculate SSH Public Key Finerprint",
|
||||
Action: "Calculate SSH Public Key Fingerprint",
|
||||
Message: err.Error(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user