diff --git a/.gitignore b/.gitignore index 500cd6776..eddb26abd 100644 --- a/.gitignore +++ b/.gitignore @@ -12,4 +12,5 @@ client apps/api/db/*.db local-serve apps/api/db/migration.db-journal -apps/api/core* \ No newline at end of file +apps/api/core* +logs \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index 958b6512f..000000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,256 +0,0 @@ -# 👋 Welcome - -First of all, thank you for considering contributing to my project! It means a lot 💜. - - -## 🙋 Want to help? - -If you begin in GitHub contribution, you can find the [first contribution](https://github.com/firstcontributions/first-contributions) and follow this guide. - -Follow the [introduction](#introduction) to get started then start contributing! - -This is a little list of what you can do to help the project: - -- [🧑‍💻 Develop your own ideas](#developer-contribution) -- [🌐 Translate the project](#translation) - -## 👋 Introduction - -### Setup with Github codespaces - -If you have github codespaces enabled then you can just create a codespace and run `pnpm dev` to run your the dev environment. All the required dependencies and packages has been configured for you already. - -### Setup with Gitpod - -If you have a [Gitpod](https://gitpod.io), you can just create a workspace from this repository, run `pnpm install && pnpm db:push && pnpm db:seed` and then `pnpm dev`. All the required dependencies and packages has been configured for you already. - -### Setup locally in your machine - -> 🔴 At the moment, Coolify **doesn't support Windows**. You must use Linux or MacOS. Consider using Gitpod or Github Codespaces. - -#### Recommended Pull Request Guideline - -- Fork the project -- Clone your fork repo to local -- Create a new branch -- Push to your fork repo -- Create a pull request: https://github.com/coollabsio/coolify/compare -- Write a proper description -- Open the pull request to review against `next` branch - ---- - -# 🧑‍💻 Developer contribution -## Technical skills required - -- **Languages**: Node.js / Javascript / Typescript -- **Framework JS/TS**: [SvelteKit](https://kit.svelte.dev/) & [Fastify](https://www.fastify.io/) -- **Database ORM**: [Prisma.io](https://www.prisma.io/) -- **Docker Engine API** - ---- - -## How to start after you set up your local fork? - -### Prerequisites -1. Due to the lock file, this repository is best with [pnpm](https://pnpm.io). I recommend you try and use `pnpm` because it is cool and efficient! - -2. You need to have [Docker Engine](https://docs.docker.com/engine/install/) installed locally. -3. You need to have [Docker Compose Plugin](https://docs.docker.com/compose/install/compose-plugin/) installed locally. -4. You need to have [GIT LFS Support](https://git-lfs.github.com/) installed locally. - -Optional: - -4. To test Heroku buildpacks, you need [pack](https://github.com/buildpacks/pack) binary installed locally. - -### Steps for local setup - -1. Copy `apps/api/.env.template` to `apps/api/.env.template` and set the `COOLIFY_APP_ID` environment variable to something cool. -2. Install dependencies with `pnpm install`. -3. Need to create a local SQlite database with `pnpm db:push`. - - This will apply all migrations at `db/dev.db`. - -4. Seed the database with base entities with `pnpm db:seed` -5. You can start coding after starting `pnpm dev`. - ---- - -## Database migrations - -During development, if you change the database layout, you need to run `pnpm db:push` to migrate the database and create types for Prisma. You also need to restart the development process. - -If the schema is finalized, you need to create a migration file with `pnpm db:migrate ` where `nameOfMigration` is given by you. Make it sense. :) - ---- - -## How to add new services - -You can add any open-source and self-hostable software (service/application) to Coolify if the following statements are true: - -- Self-hostable (obviously) -- Open-source -- Maintained (I do not want to add software full of bugs) - -## Backend - -There are 5 steps you should make on the backend side. - -1. Create Prisma / database schema for the new service. -2. Add supported versions of the service. -3. Update global functions. -4. Create API endpoints. -5. Define automatically generated variables. - -> I will use [Umami](https://umami.is/) as an example service. - -### Create Prisma / Database schema for the new service. - -You only need to do this if you store passwords or any persistent configuration. Mostly it is required by all services, but there are some exceptions, like NocoDB. - -Update Prisma schema in [prisma/schema.prisma](prisma/schema.prisma). - -- Add new model with the new service name. -- Make a relationship with `Service` model. -- In the `Service` model, the name of the new field should be with low-capital. -- If the service needs a database, define a `publicPort` field to be able to make it's database public, example field name in case of PostgreSQL: `postgresqlPublicPort`. It should be a optional field. - -If you are finished with the Prisma schema, you should update the database schema with `pnpm db:push` command. - -> You must restart the running development environment to be able to use the new model - -> If you use VSCode/TLS, you probably need to restart the `Typescript Language Server` to get the new types loaded in the running environment. - -### Add supported versions - -Supported versions are hardcoded into Coolify (for now). - -You need to update `supportedServiceTypesAndVersions` function at [apps/api/src/lib/services/supportedVersions.ts](apps/api/src/lib/services/supportedVersions.ts). Example JSON: - -```js - { - // Name used to identify the service internally - name: 'umami', - // Fancier name to show to the user - fancyName: 'Umami', - // Docker base image for the service - baseImage: 'ghcr.io/mikecao/umami', - // Optional: If there is any dependent image, you should list it here - images: [], - // Usable tags - versions: ['postgresql-latest'], - // Which tag is the recommended - recommendedVersion: 'postgresql-latest', - // Application's default port, Umami listens on 3000 - ports: { - main: 3000 - } - } -``` - -### Add required functions/properties - -1. Add the new service to the `includeServices` variable in [apps/api/src/lib/services/common.ts](apps/api/src/lib/services/common.ts), so it will be included in all places in the database queries where it is required. - -```js -const include: any = { - destinationDocker: true, - persistentStorage: true, - serviceSecret: true, - minio: true, - plausibleAnalytics: true, - vscodeserver: true, - wordpress: true, - ghost: true, - meiliSearch: true, - umami: true // This line! -}; -``` - -2. Update the database update query with the new service type to `configureServiceType` function in [apps/api/src/lib/services/common.ts](apps/api/src/lib/services/common.ts). This function defines the automatically generated variables (passwords, users, etc.) and it's encryption process (if applicable). - -```js -[...] -else if (type === 'umami') { - const postgresqlUser = cuid(); - const postgresqlPassword = encrypt(generatePassword()); - const postgresqlDatabase = 'umami'; - const hashSalt = encrypt(generatePassword(64)); - await prisma.service.update({ - where: { id }, - data: { - type, - umami: { - create: { - postgresqlDatabase, - postgresqlPassword, - postgresqlUser, - hashSalt, - } - } - } - }); - } -``` - -3. Add field details to [apps/api/src/lib/services/serviceFields.ts](apps/api/src/lib/services/serviceFields.ts), so every component will know what to do with the values (decrypt/show it by default/readonly) - -```js -export const umami = [{ - name: 'postgresqlUser', - isEditable: false, - isLowerCase: false, - isNumber: false, - isBoolean: false, - isEncrypted: false -}] -``` - -4. Add service deletion query to `removeService` function in [apps/api/src/lib/services/common.ts](apps/api/src/lib/services/common.ts) - -5. Add start process for the new service in [apps/api/src/lib/services/handlers.ts](apps/api/src/lib/services/handlers.ts) - -> See startUmamiService() function as example. - -6. Add the newly added start process to `startService` in [apps/api/src/routes/api/v1/services/handlers.ts](apps/api/src/routes/api/v1/services/handlers.ts) - -7. You need to add a custom logo at [apps/ui/src/lib/components/svg/services](apps/ui/src/lib/components/svg/services) as a svelte component and export it in [apps/ui/src/lib/components/svg/services/index.ts](apps/ui/src/lib/components/svg/services/index.ts) - - SVG is recommended, but you can use PNG as well. It should have the `isAbsolute` variable with the suitable CSS classes, primarily for sizing and positioning. - -8. You need to include it the logo at: - -- [apps/ui/src/lib/components/svg/services/ServiceIcons.svelte](apps/ui/src/lib/components/svg/services/ServiceIcons.svelte) with `isAbsolute`. -- [apps/ui/src/routes/services/[id]/_ServiceLinks.svelte](apps/ui/src/routes/services/[id]/_ServiceLinks.svelte) with the link to the docs/main site of the service - -9. By default the URL and the name frontend forms are included in [apps/ui/src/routes/services/[id]/_Services/_Services.svelte](apps/ui/src/routes/services/[id]/_Services/_Services.svelte). - - If you need to show more details on the frontend, such as users/passwords, you need to add Svelte component to [apps/ui/src/routes/services/[id]/_Services](apps/ui/src/routes/services/[id]/_Services) with an underscore. - - > For example, see other [here](apps/ui/src/routes/services/[id]/_Services/_Umami.svelte). - - -Good job! 👏 - - diff --git a/CONTRIBUTION_NEW.md b/CONTRIBUTION.md similarity index 51% rename from CONTRIBUTION_NEW.md rename to CONTRIBUTION.md index a9ed208ed..78992243b 100644 --- a/CONTRIBUTION_NEW.md +++ b/CONTRIBUTION.md @@ -1,45 +1,3 @@ ---- -head: - - - meta - - name: description - content: Coolify - Databases - - - meta - - name: keywords - content: databases coollabs coolify - - - meta - - name: twitter:card - content: summary_large_image - - - meta - - name: twitter:site - content: '@andrasbacsai' - - - meta - - name: twitter:title - content: Coolify - - - meta - - name: twitter:description - content: An open-source & self-hostable Heroku / Netlify alternative. - - - meta - - name: twitter:image - content: https://cdn.coollabs.io/assets/coollabs/og-image-databases.png - - - meta - - property: og:type - content: website - - - meta - - property: og:url - content: https://coolify.io - - - meta - - property: og:title - content: Coolify - - - meta - - property: og:description - content: An open-source & self-hostable Heroku / Netlify alternative. - - - meta - - property: og:site_name - content: Coolify - - - meta - - property: og:image - content: https://cdn.coollabs.io/assets/coollabs/og-image-databases.png ---- # Contribution First, thanks for considering to contribute to my project. It really means a lot! :) @@ -100,9 +58,58 @@ All data that needs to be persist for a service should be saved to the database very password/api key/passphrase needs to be encrypted. If you are not sure, whether it should be encrypted or not, just encrypt it. -Update Prisma schema in [src/api/prisma/schema.prisma](https://github.com/coollabsio/coolify/blob/main/apps/api/prisma/schema.prisma). +Update Prisma schema in [src/apps/api/prisma/schema.prisma](https://github.com/coollabsio/coolify/blob/main/apps/api/prisma/schema.prisma). - Add new model with the new service name. - Make a relationship with `Service` model. - In the `Service` model, the name of the new field should be with low-capital. - If the service needs a database, define a `publicPort` field to be able to make it's database public, example field name in case of PostgreSQL: `postgresqlPublicPort`. It should be a optional field. + +Once done, create Prisma schema with `pnpm db:push`. +> You may also need to restart `Typescript Language Server` in your IDE to get the new types. + +### Add available versions + +Versions are hardcoded into Coolify at the moment and based on Docker image tags. +- Update `supportedServiceTypesAndVersions` function [here](apps/api/src/lib/services/supportedVersions.ts) + +### Include the new service in queries + +At [here](apps/api/src/lib/services/common.ts) in `includeServices` function add the new table name, so it will be included in all places in the database queries where it is required. + +### Define auto-generated fields + +At [here](apps/api/src/lib/services/common.ts) in `configureServiceType` function add the initial auto-generated details such as password, users etc, and the encryption process of secrets (if applicable). + +### Define input field details + +At [here](apps/api/src/lib/services/serviceFields.ts) add details about the input fields shown in the UI, so every component (API/UI) will know what to do with the values (decrypt/show it by default/readonly/etc). + +### Define the start process + +- At [here](apps/api/src/lib/services/handlers.ts), define how the service should start. It could be complex and based on `docker-compose` definitions. + +> See `startUmamiService()` function as example. + +- At [here](apps/api/src/routes/api/v1/services/handlers.ts), add the new start service process to `startService` function. + +### Define the deletion process + +[Here](apps/api/src/lib/services/common.ts) in `removeService` add the database deletion process. + +### Custom logo + +- At [here](apps/ui/src/lib/components/svg/services) add the service custom log as a Svelte component and export it [here](apps/ui/src/lib/components/svg/services/index.ts). + +> SVG is recommended, but you can use PNG as well. It should have the `isAbsolute` variable with the suitable CSS classes, primarily for sizing and positioning. + +- At [here](apps/ui/src/lib/components/svg/services/ServiceIcons.svelte) include the new logo with `isAbsolute` property. + +- At [here](apps/ui/src/routes/services/[id]/_ServiceLinks.svelte) add links to the documentation of the service. + +### Custom fields on the UI +By default the URL and name are shown on the UI. Everything else needs to be added [here](apps/ui/src/routes/services/[id]/_Services/_Services.svelte) + +> If you need to show more details on the frontend, such as users/passwords, you need to add Svelte component [here](apps/ui/src/routes/services/[id]/_Services) with an underscore. For example, see other [here](apps/ui/src/routes/services/[id]/_Services/_Umami.svelte). + +Good job! 👏 \ No newline at end of file diff --git a/Dockerfile b/Dockerfile index 675fe5b5b..b7bd56212 100644 --- a/Dockerfile +++ b/Dockerfile @@ -33,6 +33,7 @@ RUN chmod +x ~/.docker/cli-plugins/docker-compose /usr/bin/docker RUN (curl -sSL "https://github.com/buildpacks/pack/releases/download/v0.27.0/pack-v0.27.0-linux.tgz" | tar -C /usr/local/bin/ --no-same-owner -xzv pack) COPY --from=build /app/apps/api/build/ . +COPY --from=build /app/others/fluentbit/ ./fluentbit COPY --from=build /app/apps/ui/build/ ./public COPY --from=build /app/apps/api/prisma/ ./prisma COPY --from=build /app/apps/api/package.json . diff --git a/apps/api/package.json b/apps/api/package.json index f436482c8..f3635517b 100644 --- a/apps/api/package.json +++ b/apps/api/package.json @@ -29,6 +29,8 @@ "bree": "9.1.2", "cabin": "9.1.2", "compare-versions": "5.0.1", + "csv-parse": "^5.3.0", + "csvtojson": "^2.0.10", "cuid": "2.1.8", "dayjs": "1.11.5", "dockerode": "3.3.4", diff --git a/apps/api/prisma/migrations/20220913092100_preview_applications/migration.sql b/apps/api/prisma/migrations/20220913092100_preview_applications/migration.sql new file mode 100644 index 000000000..0ec1aafa0 --- /dev/null +++ b/apps/api/prisma/migrations/20220913092100_preview_applications/migration.sql @@ -0,0 +1,18 @@ +-- AlterTable +ALTER TABLE "Build" ADD COLUMN "previewApplicationId" TEXT; + +-- CreateTable +CREATE TABLE "PreviewApplication" ( + "id" TEXT NOT NULL PRIMARY KEY, + "pullmergeRequestId" TEXT NOT NULL, + "sourceBranch" TEXT NOT NULL, + "isRandomDomain" BOOLEAN NOT NULL DEFAULT false, + "customDomain" TEXT, + "applicationId" TEXT NOT NULL, + "createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" DATETIME NOT NULL, + CONSTRAINT "PreviewApplication_applicationId_fkey" FOREIGN KEY ("applicationId") REFERENCES "Application" ("id") ON DELETE RESTRICT ON UPDATE CASCADE +); + +-- CreateIndex +CREATE UNIQUE INDEX "PreviewApplication_applicationId_key" ON "PreviewApplication"("applicationId"); diff --git a/apps/api/prisma/schema.prisma b/apps/api/prisma/schema.prisma index df7516e01..a7500539d 100644 --- a/apps/api/prisma/schema.prisma +++ b/apps/api/prisma/schema.prisma @@ -119,6 +119,19 @@ model Application { secrets Secret[] teams Team[] connectedDatabase ApplicationConnectedDatabase? + previewApplication PreviewApplication[] +} + +model PreviewApplication { + id String @id @default(cuid()) + pullmergeRequestId String + sourceBranch String + isRandomDomain Boolean @default(false) + customDomain String? + applicationId String @unique + application Application @relation(fields: [applicationId], references: [id]) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt } model ApplicationConnectedDatabase { @@ -210,21 +223,22 @@ model BuildLog { } model Build { - id String @id @default(cuid()) - type String - applicationId String? - destinationDockerId String? - gitSourceId String? - githubAppId String? - gitlabAppId String? - commit String? - pullmergeRequestId String? - forceRebuild Boolean @default(false) - sourceBranch String? - branch String? - status String? @default("queued") - createdAt DateTime @default(now()) - updatedAt DateTime @updatedAt + id String @id @default(cuid()) + type String + applicationId String? + destinationDockerId String? + gitSourceId String? + githubAppId String? + gitlabAppId String? + commit String? + pullmergeRequestId String? + previewApplicationId String? + forceRebuild Boolean @default(false) + sourceBranch String? + branch String? + status String? @default("queued") + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt } model DestinationDocker { diff --git a/apps/api/src/jobs/deployApplication.ts b/apps/api/src/jobs/deployApplication.ts index 23adbf608..55a7ef461 100644 --- a/apps/api/src/jobs/deployApplication.ts +++ b/apps/api/src/jobs/deployApplication.ts @@ -38,8 +38,16 @@ import * as buildpacks from '../lib/buildPacks'; for (const queueBuild of queuedBuilds) { actions.push(async () => { let application = await prisma.application.findUnique({ where: { id: queueBuild.applicationId }, include: { destinationDocker: true, gitSource: { include: { githubApp: true, gitlabApp: true } }, persistentStorage: true, secrets: true, settings: true, teams: true } }) - let { id: buildId, type, sourceBranch = null, pullmergeRequestId = null, forceRebuild } = queueBuild + let { id: buildId, type, sourceBranch = null, pullmergeRequestId = null, previewApplicationId = null, forceRebuild } = queueBuild application = decryptApplication(application) + const originalApplicationId = application.id + if (pullmergeRequestId) { + const previewApplications = await prisma.previewApplication.findMany({ where: { applicationId: originalApplicationId, pullmergeRequestId } }) + if (previewApplications.length > 0) { + previewApplicationId = previewApplications[0].id + } + } + const usableApplicationId = previewApplicationId || originalApplicationId try { if (queueBuild.status === 'running') { await saveBuildLog({ line: 'Building halted, restarting...', buildId, applicationId: application.id }); @@ -104,17 +112,17 @@ import * as buildpacks from '../lib/buildPacks'; ) .digest('hex'); const { debug } = settings; - if (concurrency === 1) { - await prisma.build.updateMany({ - where: { - status: { in: ['queued', 'running'] }, - id: { not: buildId }, - applicationId, - createdAt: { lt: new Date(new Date().getTime() - 10 * 1000) } - }, - data: { status: 'failed' } - }); - } + // if (concurrency === 1) { + // await prisma.build.updateMany({ + // where: { + // status: { in: ['queued', 'running'] }, + // id: { not: buildId }, + // applicationId, + // createdAt: { lt: new Date(new Date().getTime() - 10 * 1000) } + // }, + // data: { status: 'failed' } + // }); + // } let imageId = applicationId; let domain = getDomain(fqdn); const volumes = @@ -338,10 +346,15 @@ import * as buildpacks from '../lib/buildPacks'; await saveBuildLog({ line: 'Deployment successful!', buildId, applicationId }); } catch (error) { await saveBuildLog({ line: error, buildId, applicationId }); - await prisma.build.updateMany({ - where: { id: buildId, status: { in: ['queued', 'running'] } }, - data: { status: 'failed' } - }); + const foundBuild = await prisma.build.findUnique({ where: { id: buildId } }) + if (foundBuild) { + await prisma.build.update({ + where: { id: buildId }, + data: { + status: 'failed' + } + }); + } throw new Error(error); } await saveBuildLog({ line: 'Proxy will be updated shortly.', buildId, applicationId }); @@ -353,10 +366,15 @@ import * as buildpacks from '../lib/buildPacks'; } } catch (error) { - await prisma.build.updateMany({ - where: { id: buildId, status: { in: ['queued', 'running'] } }, - data: { status: 'failed' } - }); + const foundBuild = await prisma.build.findUnique({ where: { id: buildId } }) + if (foundBuild) { + await prisma.build.update({ + where: { id: buildId }, + data: { + status: 'failed' + } + }); + } if (error !== 1) { await saveBuildLog({ line: error, buildId, applicationId: application.id }); } diff --git a/apps/api/src/jobs/infrastructure.ts b/apps/api/src/jobs/infrastructure.ts index 178a06aca..23e380610 100644 --- a/apps/api/src/jobs/infrastructure.ts +++ b/apps/api/src/jobs/infrastructure.ts @@ -29,7 +29,7 @@ async function autoUpdater() { `sed -i '/COOLIFY_AUTO_UPDATE=/cCOOLIFY_AUTO_UPDATE=${isAutoUpdateEnabled}' .env` ); await asyncExecShell( - `docker run --rm -tid --env-file .env -v /var/run/docker.sock:/var/run/docker.sock -v coolify-db coollabsio/coolify:${latestVersion} /bin/sh -c "env | grep COOLIFY > .env && echo 'TAG=${latestVersion}' >> .env && docker stop -t 0 coolify && docker rm coolify && docker compose up -d --force-recreate"` + `docker run --rm -tid --env-file .env -v /var/run/docker.sock:/var/run/docker.sock -v coolify-db coollabsio/coolify:${latestVersion} /bin/sh -c "env | grep COOLIFY > .env && echo 'TAG=${latestVersion}' >> .env && docker stop -t 0 coolify coolify-fluentbit && docker rm coolify coolify-fluentbit && docker compose pull && docker compose up -d --force-recreate"` ); } } else { diff --git a/apps/api/src/lib/buildPacks/common.ts b/apps/api/src/lib/buildPacks/common.ts index 0e188ae36..44812a76d 100644 --- a/apps/api/src/lib/buildPacks/common.ts +++ b/apps/api/src/lib/buildPacks/common.ts @@ -1,4 +1,4 @@ -import { base64Encode, executeDockerCmd, generateTimestamp, getDomain, isDev, prisma, version } from "../common"; +import { base64Encode, encrypt, executeDockerCmd, generateTimestamp, getDomain, isDev, prisma, version } from "../common"; import { promises as fs } from 'fs'; import { day } from "../dayjs"; @@ -461,17 +461,32 @@ export const saveBuildLog = async ({ buildId: string; applicationId: string; }): Promise => { + const { default: got } = await import('got') + if (line && typeof line === 'string' && line.includes('ghs_')) { const regex = /ghs_.*@/g; line = line.replace(regex, '@'); } const addTimestamp = `[${generateTimestamp()}] ${line}`; - if (isDev) console.debug(`[${applicationId}] ${addTimestamp}`); - return await prisma.buildLog.create({ - data: { - line: addTimestamp, buildId, time: Number(day().valueOf()), applicationId - } - }); + const fluentBitUrl = isDev ? 'http://localhost:24224' : 'http://coolify-fluentbit:24224'; + + if (isDev) { + console.debug(`[${applicationId}] ${addTimestamp}`); + } + try { + return await got.post(`${fluentBitUrl}/${applicationId}_buildlog_${buildId}.csv`, { + json: { + line: encrypt(line) + } + }) + } catch(error) { + return await prisma.buildLog.create({ + data: { + line: addTimestamp, buildId, time: Number(day().valueOf()), applicationId + } + }); + } + }; export async function copyBaseConfigurationFiles( @@ -707,7 +722,6 @@ export async function buildCacheImageWithNode(data, imageForBuild) { Dockerfile.push(`RUN ${installCommand}`); } Dockerfile.push(`RUN ${buildCommand}`); - console.log(Dockerfile.join('\n')) await fs.writeFile(`${workdir}/Dockerfile-cache`, Dockerfile.join('\n')); await buildImage({ ...data, isCache: true }); } diff --git a/apps/api/src/lib/common.ts b/apps/api/src/lib/common.ts index 857bb1910..cb2b10e17 100644 --- a/apps/api/src/lib/common.ts +++ b/apps/api/src/lib/common.ts @@ -21,7 +21,7 @@ import { scheduler } from './scheduler'; import { supportedServiceTypesAndVersions } from './services/supportedVersions'; import { includeServices } from './services/common'; -export const version = '3.10.3'; +export const version = '3.10.4'; export const isDev = process.env.NODE_ENV === 'development'; const algorithm = 'aes-256-ctr'; @@ -45,7 +45,7 @@ export function getAPIUrl() { if (process.env.CODESANDBOX_HOST) { return `https://${process.env.CODESANDBOX_HOST.replace(/\$PORT/, '3001')}` } - return isDev ? 'http://localhost:3001' : 'http://localhost:3000'; + return isDev ? 'http://host.docker.internal:3001' : 'http://localhost:3000'; } export function getUIUrl() { diff --git a/apps/api/src/lib/services/handlers.ts b/apps/api/src/lib/services/handlers.ts index 6a5bfe91a..f6aaa6be5 100644 --- a/apps/api/src/lib/services/handlers.ts +++ b/apps/api/src/lib/services/handlers.ts @@ -1374,10 +1374,6 @@ async function startAppWriteService(request: FastifyRequest) { const teamId = request.user.teamId; const { version, fqdn, destinationDocker, secrets, exposePort, network, port, workdir, image, appwrite } = await defaultServiceConfigurations({ id, teamId }) - let isStatsEnabled = false - if (secrets.find(s => s === '_APP_USAGE_STATS=enabled')) { - isStatsEnabled = true - } const { opensslKeyV1, executorSecret, @@ -1755,50 +1751,48 @@ async function startAppWriteService(request: FastifyRequest) { }, }; - if (isStatsEnabled) { - dockerCompose[id].depends_on.push(`${id}-influxdb`); - dockerCompose[`${id}-usage`] = { - image: `${image}:${version}`, - container_name: `${id}-usage`, - labels: makeLabelForServices('appwrite'), - entrypoint: "usage", - depends_on: [ - `${id}-mariadb`, - `${id}-influxdb`, - ], - environment: [ - "_APP_ENV=production", - `_APP_OPENSSL_KEY_V1=${opensslKeyV1}`, - `_APP_DB_HOST=${mariadbHost}`, - `_APP_DB_PORT=${mariadbPort}`, - `_APP_DB_SCHEMA=${mariadbDatabase}`, - `_APP_DB_USER=${mariadbUser}`, - `_APP_DB_PASS=${mariadbPassword}`, - `_APP_INFLUXDB_HOST=${id}-influxdb`, - "_APP_INFLUXDB_PORT=8086", - `_APP_REDIS_HOST=${id}-redis`, - "_APP_REDIS_PORT=6379", - ...secrets - ], - ...defaultComposeConfiguration(network), - } - dockerCompose[`${id}-influxdb`] = { - image: "appwrite/influxdb:1.5.0", - container_name: `${id}-influxdb`, - volumes: [ - `${id}-influxdb:/var/lib/influxdb:rw` - ], - ...defaultComposeConfiguration(network), - } - dockerCompose[`${id}-telegraf`] = { - image: "appwrite/telegraf:1.4.0", - container_name: `${id}-telegraf`, - environment: [ - `_APP_INFLUXDB_HOST=${id}-influxdb`, - "_APP_INFLUXDB_PORT=8086", - ], - ...defaultComposeConfiguration(network), - } + dockerCompose[id].depends_on.push(`${id}-influxdb`); + dockerCompose[`${id}-usage`] = { + image: `${image}:${version}`, + container_name: `${id}-usage`, + labels: makeLabelForServices('appwrite'), + entrypoint: "usage", + depends_on: [ + `${id}-mariadb`, + `${id}-influxdb`, + ], + environment: [ + "_APP_ENV=production", + `_APP_OPENSSL_KEY_V1=${opensslKeyV1}`, + `_APP_DB_HOST=${mariadbHost}`, + `_APP_DB_PORT=${mariadbPort}`, + `_APP_DB_SCHEMA=${mariadbDatabase}`, + `_APP_DB_USER=${mariadbUser}`, + `_APP_DB_PASS=${mariadbPassword}`, + `_APP_INFLUXDB_HOST=${id}-influxdb`, + "_APP_INFLUXDB_PORT=8086", + `_APP_REDIS_HOST=${id}-redis`, + "_APP_REDIS_PORT=6379", + ...secrets + ], + ...defaultComposeConfiguration(network), + } + dockerCompose[`${id}-influxdb`] = { + image: "appwrite/influxdb:1.5.0", + container_name: `${id}-influxdb`, + volumes: [ + `${id}-influxdb:/var/lib/influxdb:rw` + ], + ...defaultComposeConfiguration(network), + } + dockerCompose[`${id}-telegraf`] = { + image: "appwrite/telegraf:1.4.0", + container_name: `${id}-telegraf`, + environment: [ + `_APP_INFLUXDB_HOST=${id}-influxdb`, + "_APP_INFLUXDB_PORT=8086", + ], + ...defaultComposeConfiguration(network), } const composeFile: any = { diff --git a/apps/api/src/lib/services/supportedVersions.ts b/apps/api/src/lib/services/supportedVersions.ts index f4a08e5a2..643215105 100644 --- a/apps/api/src/lib/services/supportedVersions.ts +++ b/apps/api/src/lib/services/supportedVersions.ts @@ -1,3 +1,24 @@ +/* + Example of a supported version: +{ + // Name used to identify the service internally + name: 'umami', + // Fancier name to show to the user + fancyName: 'Umami', + // Docker base image for the service + baseImage: 'ghcr.io/mikecao/umami', + // Optional: If there is any dependent image, you should list it here + images: [], + // Usable tags + versions: ['postgresql-latest'], + // Which tag is the recommended + recommendedVersion: 'postgresql-latest', + // Application's default port, Umami listens on 3000 + ports: { + main: 3000 + } + } +*/ export const supportedServiceTypesAndVersions = [ { name: 'plausibleanalytics', @@ -151,7 +172,7 @@ export const supportedServiceTypesAndVersions = [ fancyName: 'Appwrite', baseImage: 'appwrite/appwrite', images: ['mariadb:10.7', 'redis:6.2-alpine', 'appwrite/telegraf:1.4.0'], - versions: ['latest', '0.15.3'], + versions: ['latest', '1.0','0.15.3'], recommendedVersion: '0.15.3', ports: { main: 80 diff --git a/apps/api/src/routes/api/v1/applications/handlers.ts b/apps/api/src/routes/api/v1/applications/handlers.ts index c2b375c2a..ffb9e64d4 100644 --- a/apps/api/src/routes/api/v1/applications/handlers.ts +++ b/apps/api/src/routes/api/v1/applications/handlers.ts @@ -5,6 +5,7 @@ import axios from 'axios'; import { FastifyReply } from 'fastify'; import fs from 'fs/promises'; import yaml from 'js-yaml'; +import csv from 'csvtojson'; import { day } from '../../../../lib/dayjs'; import { makeLabelForStandaloneApplication, setDefaultBaseImage, setDefaultConfiguration } from '../../../../lib/buildPacks/common'; @@ -12,8 +13,9 @@ import { checkDomainsIsValidInDNS, checkDoubleBranch, checkExposedPort, createDi import { checkContainer, formatLabelsOnDocker, isContainerExited, removeContainer } from '../../../../lib/docker'; import type { FastifyRequest } from 'fastify'; -import type { GetImages, CancelDeployment, CheckDNS, CheckRepository, DeleteApplication, DeleteSecret, DeleteStorage, GetApplicationLogs, GetBuildIdLogs, GetBuildLogs, SaveApplication, SaveApplicationSettings, SaveApplicationSource, SaveDeployKey, SaveDestination, SaveSecret, SaveStorage, DeployApplication, CheckDomain, StopPreviewApplication } from './types'; +import type { GetImages, CancelDeployment, CheckDNS, CheckRepository, DeleteApplication, DeleteSecret, DeleteStorage, GetApplicationLogs, GetBuildIdLogs, SaveApplication, SaveApplicationSettings, SaveApplicationSource, SaveDeployKey, SaveDestination, SaveSecret, SaveStorage, DeployApplication, CheckDomain, StopPreviewApplication, RestartPreviewApplication, GetBuilds } from './types'; import { OnlyId } from '../../../../types'; +import path from 'node:path'; function filterObject(obj, callback) { return Object.fromEntries(Object.entries(obj). @@ -83,8 +85,6 @@ export async function getApplicationStatus(request: FastifyRequest) { isExited = status.status.isExited; isRestarting = status.status.isRestarting } - - // isExited = await isContainerExited(application.destinationDocker.id, id); } return { isRunning, @@ -164,7 +164,8 @@ export async function getApplicationFromDB(id: string, teamId: string) { gitSource: { include: { githubApp: true, gitlabApp: true } }, secrets: true, persistentStorage: true, - connectedDatabase: true + connectedDatabase: true, + previewApplication: true } }); if (!application) { @@ -350,6 +351,7 @@ export async function stopPreviewApplication(request: FastifyRequest, reply: Fas try { const { id } = request.params let { name, value, isBuildSecret, isPRMRSecret, isNew } = request.body - if (isNew) { const found = await prisma.secret.findFirst({ where: { name, applicationId: id, isPRMRSecret } }); if (found) { @@ -820,14 +821,24 @@ export async function saveSecret(request: FastifyRequest, reply: Fas }); } } else { - value = encrypt(value.trim()); + if (value) { + value = encrypt(value.trim()); + } const found = await prisma.secret.findFirst({ where: { applicationId: id, name, isPRMRSecret } }); if (found) { - await prisma.secret.updateMany({ - where: { applicationId: id, name, isPRMRSecret }, - data: { value, isBuildSecret, isPRMRSecret } - }); + if (!value && isPRMRSecret) { + await prisma.secret.deleteMany({ + where: { applicationId: id, name, isPRMRSecret } + }); + } else { + + await prisma.secret.updateMany({ + where: { applicationId: id, name, isPRMRSecret }, + data: { value, isBuildSecret, isPRMRSecret } + }); + } + } else { await prisma.secret.create({ data: { name, value, isBuildSecret, isPRMRSecret, application: { connect: { id } } } @@ -894,6 +905,181 @@ export async function deleteStorage(request: FastifyRequest) { } } +export async function restartPreview(request: FastifyRequest, reply: FastifyReply) { + try { + const { id, pullmergeRequestId } = request.params + const { teamId } = request.user + let application: any = await getApplicationFromDB(id, teamId); + if (application?.destinationDockerId) { + const buildId = cuid(); + const { id: dockerId, network } = application.destinationDocker; + const { secrets, port, repository, persistentStorage, id: applicationId, buildPack, exposePort } = application; + + const envs = [ + `PORT=${port}` + ]; + if (secrets.length > 0) { + secrets.forEach((secret) => { + if (pullmergeRequestId) { + const isSecretFound = secrets.filter(s => s.name === secret.name && s.isPRMRSecret) + if (isSecretFound.length > 0) { + envs.push(`${secret.name}=${isSecretFound[0].value}`); + } else { + envs.push(`${secret.name}=${secret.value}`); + } + } else { + if (!secret.isPRMRSecret) { + envs.push(`${secret.name}=${secret.value}`); + } + } + }); + } + const { workdir } = await createDirectories({ repository, buildId }); + const labels = [] + let image = null + const { stdout: container } = await executeDockerCmd({ dockerId, command: `docker container ls --filter 'label=com.docker.compose.service=${id}-${pullmergeRequestId}' --format '{{json .}}'` }) + const containersArray = container.trim().split('\n'); + for (const container of containersArray) { + const containerObj = formatLabelsOnDocker(container); + image = containerObj[0].Image + Object.keys(containerObj[0].Labels).forEach(function (key) { + if (key.startsWith('coolify')) { + labels.push(`${key}=${containerObj[0].Labels[key]}`) + } + }) + } + let imageFound = false; + try { + await executeDockerCmd({ + dockerId, + command: `docker image inspect ${image}` + }) + imageFound = true; + } catch (error) { + // + } + if (!imageFound) { + throw { status: 500, message: 'Image not found, cannot restart application.' } + } + await fs.writeFile(`${workdir}/.env`, envs.join('\n')); + + let envFound = false; + try { + envFound = !!(await fs.stat(`${workdir}/.env`)); + } catch (error) { + // + } + const volumes = + persistentStorage?.map((storage) => { + return `${applicationId}${storage.path.replace(/\//gi, '-')}:${buildPack !== 'docker' ? '/app' : '' + }${storage.path}`; + }) || []; + const composeVolumes = volumes.map((volume) => { + return { + [`${volume.split(':')[0]}`]: { + name: volume.split(':')[0] + } + }; + }); + const composeFile = { + version: '3.8', + services: { + [`${applicationId}-${pullmergeRequestId}`]: { + image, + container_name: `${applicationId}-${pullmergeRequestId}`, + volumes, + env_file: envFound ? [`${workdir}/.env`] : [], + labels, + depends_on: [], + expose: [port], + ...(exposePort ? { ports: [`${exposePort}:${port}`] } : {}), + ...defaultComposeConfiguration(network), + } + }, + networks: { + [network]: { + external: true + } + }, + volumes: Object.assign({}, ...composeVolumes) + }; + await fs.writeFile(`${workdir}/docker-compose.yml`, yaml.dump(composeFile)); + await executeDockerCmd({ dockerId, command: `docker stop -t 0 ${id}-${pullmergeRequestId}` }) + await executeDockerCmd({ dockerId, command: `docker rm ${id}-${pullmergeRequestId}` }) + await executeDockerCmd({ dockerId, command: `docker compose --project-directory ${workdir} up -d` }) + return reply.code(201).send(); + } + throw { status: 500, message: 'Application cannot be restarted.' } + } catch ({ status, message }) { + return errorHandler({ status, message }) + } +} +export async function getPreviewStatus(request: FastifyRequest) { + try { + const { id, pullmergeRequestId } = request.params + const { teamId } = request.user + let isRunning = false; + let isExited = false; + let isRestarting = false; + let isBuilding = false + const application: any = await getApplicationFromDB(id, teamId); + if (application?.destinationDockerId) { + const status = await checkContainer({ dockerId: application.destinationDocker.id, container: `${id}-${pullmergeRequestId}` }); + if (status?.found) { + isRunning = status.status.isRunning; + isExited = status.status.isExited; + isRestarting = status.status.isRestarting + } + const building = await prisma.build.findMany({ where: { applicationId: id, pullmergeRequestId, status: { in: ['queued', 'running'] } } }) + isBuilding = building.length > 0 + } + return { + isBuilding, + isRunning, + isRestarting, + isExited, + }; + } catch ({ status, message }) { + return errorHandler({ status, message }) + } +} +export async function loadPreviews(request: FastifyRequest) { + try { + const { id } = request.params + const application = await prisma.application.findUnique({ where: { id }, include: { destinationDocker: true } }); + const { stdout } = await executeDockerCmd({ dockerId: application.destinationDocker.id, command: `docker container ls --filter 'name=${id}-' --format "{{json .}}"` }) + if (stdout === '') { + throw { status: 500, message: 'No previews found.' } + } + const containers = formatLabelsOnDocker(stdout).filter(container => container.Labels['coolify.configuration'] && container.Labels['coolify.type'] === 'standalone-application') + + const jsonContainers = containers + .map((container) => + JSON.parse(Buffer.from(container.Labels['coolify.configuration'], 'base64').toString()) + ) + .filter((container) => { + return container.pullmergeRequestId && container.applicationId === id; + }); + for (const container of jsonContainers) { + const found = await prisma.previewApplication.findMany({ where: { applicationId: container.applicationId, pullmergeRequestId: container.pullmergeRequestId } }) + if (found.length === 0) { + await prisma.previewApplication.create({ + data: { + pullmergeRequestId: container.pullmergeRequestId, + sourceBranch: container.branch, + customDomain: container.fqdn, + application: { connect: { id: container.applicationId } } + } + }) + } + } + return { + previews: await prisma.previewApplication.findMany({ where: { applicationId: id } }) + } + } catch ({ status, message }) { + return errorHandler({ status, message }) + } +} export async function getPreviews(request: FastifyRequest) { try { const { id } = request.params @@ -909,26 +1095,7 @@ export async function getPreviews(request: FastifyRequest) { const applicationSecrets = secrets.filter((secret) => !secret.isPRMRSecret); const PRMRSecrets = secrets.filter((secret) => secret.isPRMRSecret); - const application = await prisma.application.findUnique({ where: { id }, include: { destinationDocker: true } }); - const { stdout } = await executeDockerCmd({ dockerId: application.destinationDocker.id, command: `docker container ls --filter 'name=${id}-' --format "{{json .}}"` }) - if (stdout === '') { - return { - containers: [], - applicationSecrets: [], - PRMRSecrets: [] - } - } - const containers = formatLabelsOnDocker(stdout).filter(container => container.Labels['coolify.configuration'] && container.Labels['coolify.type'] === 'standalone-application') - - const jsonContainers = containers - .map((container) => - JSON.parse(Buffer.from(container.Labels['coolify.configuration'], 'base64').toString()) - ) - .filter((container) => { - return container.pullmergeRequestId && container.applicationId === id; - }); return { - containers: jsonContainers, applicationSecrets: applicationSecrets.sort((a, b) => { return ('' + a.name).localeCompare(b.name); }), @@ -980,7 +1147,7 @@ export async function getApplicationLogs(request: FastifyRequest) { +export async function getBuilds(request: FastifyRequest) { try { const { id } = request.params let { buildId, skip = 0 } = request.query @@ -997,17 +1164,15 @@ export async function getBuildLogs(request: FastifyRequest) { builds = await prisma.build.findMany({ where: { applicationId: id }, orderBy: { createdAt: 'desc' }, - take: 5, - skip + take: 5 + skip }); } - builds = builds.map((build) => { - const updatedAt = day(build.updatedAt).utc(); - build.took = updatedAt.diff(day(build.createdAt)) / 1000; - build.since = updatedAt.fromNow(); - return build; - }); + if (build.status === 'running') { + build.elapsed = (day().utc().diff(day(build.createdAt)) / 1000).toFixed(0); + } + return build + }) return { builds, buildCount @@ -1019,22 +1184,49 @@ export async function getBuildLogs(request: FastifyRequest) { export async function getBuildIdLogs(request: FastifyRequest) { try { - const { buildId } = request.params + // TODO: Fluentbit could still hold the logs, so we need to check if the logs are done + const { buildId, id } = request.params let { sequence = 0 } = request.query if (typeof sequence !== 'number') { sequence = Number(sequence) } - let logs = await prisma.buildLog.findMany({ - where: { buildId, time: { gt: sequence } }, - orderBy: { time: 'asc' } - }); + let file = `/app/logs/${id}_buildlog_${buildId}.csv` + if (isDev) { + file = `${process.cwd()}/../../logs/${id}_buildlog_${buildId}.csv` + } const data = await prisma.build.findFirst({ where: { id: buildId } }); const createdAt = day(data.createdAt).utc(); + try { + await fs.stat(file) + } catch (error) { + let logs = await prisma.buildLog.findMany({ + where: { buildId, time: { gt: sequence } }, + orderBy: { time: 'asc' } + }); + const data = await prisma.build.findFirst({ where: { id: buildId } }); + const createdAt = day(data.createdAt).utc(); + return { + logs: logs.map(log => { + log.time = Number(log.time) + return log + }), + fromDb: true, + took: day().diff(createdAt) / 1000, + status: data?.status || 'queued' + } + } + let fileLogs = (await fs.readFile(file)).toString() + let decryptedLogs = await csv({ noheader: true }).fromString(fileLogs) + let logs = decryptedLogs.map(log => { + const parsed = { + time: log['field1'], + line: decrypt(log['field2'] + '","' + log['field3']) + } + return parsed + }).filter(log => log.time > sequence) return { - logs: logs.map(log => { - log.time = Number(log.time) - return log - }), + logs, + fromDb: false, took: day().diff(createdAt) / 1000, status: data?.status || 'queued' } diff --git a/apps/api/src/routes/api/v1/applications/index.ts b/apps/api/src/routes/api/v1/applications/index.ts index d42906066..74370de4d 100644 --- a/apps/api/src/routes/api/v1/applications/index.ts +++ b/apps/api/src/routes/api/v1/applications/index.ts @@ -1,8 +1,8 @@ import { FastifyPluginAsync } from 'fastify'; import { OnlyId } from '../../../../types'; -import { cancelDeployment, checkDNS, checkDomain, checkRepository, deleteApplication, deleteSecret, deleteStorage, deployApplication, getApplication, getApplicationLogs, getApplicationStatus, getBuildIdLogs, getBuildLogs, getBuildPack, getGitHubToken, getGitLabSSHKey, getImages, getPreviews, getSecrets, getStorages, getUsage, listApplications, newApplication, restartApplication, saveApplication, saveApplicationSettings, saveApplicationSource, saveBuildPack, saveConnectedDatabase, saveDeployKey, saveDestination, saveGitLabSSHKey, saveRepository, saveSecret, saveStorage, stopApplication, stopPreviewApplication } from './handlers'; +import { cancelDeployment, checkDNS, checkDomain, checkRepository, deleteApplication, deleteSecret, deleteStorage, deployApplication, getApplication, getApplicationLogs, getApplicationStatus, getBuildIdLogs, getBuildPack, getBuilds, getGitHubToken, getGitLabSSHKey, getImages, getPreviews, getPreviewStatus, getSecrets, getStorages, getUsage, listApplications, loadPreviews, newApplication, restartApplication, restartPreview, saveApplication, saveApplicationSettings, saveApplicationSource, saveBuildPack, saveConnectedDatabase, saveDeployKey, saveDestination, saveGitLabSSHKey, saveRepository, saveSecret, saveStorage, stopApplication, stopPreviewApplication } from './handlers'; -import type { CancelDeployment, CheckDNS, CheckDomain, CheckRepository, DeleteApplication, DeleteSecret, DeleteStorage, DeployApplication, GetApplicationLogs, GetBuildIdLogs, GetBuildLogs, GetImages, SaveApplication, SaveApplicationSettings, SaveApplicationSource, SaveDeployKey, SaveDestination, SaveSecret, SaveStorage, StopPreviewApplication } from './types'; +import type { CancelDeployment, CheckDNS, CheckDomain, CheckRepository, DeleteApplication, DeleteSecret, DeleteStorage, DeployApplication, GetApplicationLogs, GetBuildIdLogs, GetBuilds, GetImages, RestartPreviewApplication, SaveApplication, SaveApplicationSettings, SaveApplicationSource, SaveDeployKey, SaveDestination, SaveSecret, SaveStorage, StopPreviewApplication } from './types'; const root: FastifyPluginAsync = async (fastify): Promise => { fastify.addHook('onRequest', async (request) => { @@ -37,9 +37,12 @@ const root: FastifyPluginAsync = async (fastify): Promise => { fastify.delete('/:id/storages', async (request) => await deleteStorage(request)); fastify.get('/:id/previews', async (request) => await getPreviews(request)); + fastify.post('/:id/previews/load', async (request) => await loadPreviews(request)); + fastify.get('/:id/previews/:pullmergeRequestId/status', async (request) => await getPreviewStatus(request)); + fastify.post('/:id/previews/:pullmergeRequestId/restart', async (request, reply) => await restartPreview(request, reply)); fastify.get('/:id/logs', async (request) => await getApplicationLogs(request)); - fastify.get('/:id/logs/build', async (request) => await getBuildLogs(request)); + fastify.get('/:id/logs/build', async (request) => await getBuilds(request)); fastify.get('/:id/logs/build/:buildId', async (request) => await getBuildIdLogs(request)); fastify.get('/:id/usage', async (request) => await getUsage(request)) diff --git a/apps/api/src/routes/api/v1/applications/types.ts b/apps/api/src/routes/api/v1/applications/types.ts index e88a79d72..0699518a5 100644 --- a/apps/api/src/routes/api/v1/applications/types.ts +++ b/apps/api/src/routes/api/v1/applications/types.ts @@ -89,7 +89,7 @@ export interface GetApplicationLogs extends OnlyId { since: number, } } -export interface GetBuildLogs extends OnlyId { +export interface GetBuilds extends OnlyId { Querystring: { buildId: string skip: number, @@ -97,6 +97,7 @@ export interface GetBuildLogs extends OnlyId { } export interface GetBuildIdLogs { Params: { + id: string, buildId: string }, Querystring: { @@ -126,4 +127,10 @@ export interface StopPreviewApplication extends OnlyId { Body: { pullmergeRequestId: string | null, } +} +export interface RestartPreviewApplication { + Params: { + id: string, + pullmergeRequestId: string | null, + } } \ No newline at end of file diff --git a/apps/api/src/routes/api/v1/handlers.ts b/apps/api/src/routes/api/v1/handlers.ts index a0217c58a..7b0e9959b 100644 --- a/apps/api/src/routes/api/v1/handlers.ts +++ b/apps/api/src/routes/api/v1/handlers.ts @@ -73,7 +73,7 @@ export async function update(request: FastifyRequest) { `sed -i '/COOLIFY_AUTO_UPDATE=/cCOOLIFY_AUTO_UPDATE=${isAutoUpdateEnabled}' .env` ); await asyncExecShell( - `docker run --rm -tid --env-file .env -v /var/run/docker.sock:/var/run/docker.sock -v coolify-db coollabsio/coolify:${latestVersion} /bin/sh -c "env | grep COOLIFY > .env && echo 'TAG=${latestVersion}' >> .env && docker stop -t 0 coolify && docker rm coolify && docker compose up -d --force-recreate"` + `docker run --rm -tid --env-file .env -v /var/run/docker.sock:/var/run/docker.sock -v coolify-db coollabsio/coolify:${latestVersion} /bin/sh -c "env | grep COOLIFY > .env && echo 'TAG=${latestVersion}' >> .env && docker stop -t 0 coolify coolify-fluentbit && docker rm coolify coolify-fluentbit && docker compose pull && docker compose up -d --force-recreate"` ); return {}; } else { diff --git a/apps/api/src/routes/api/v1/servers/handlers.ts b/apps/api/src/routes/api/v1/servers/handlers.ts index c917836d2..d9705e9ef 100644 --- a/apps/api/src/routes/api/v1/servers/handlers.ts +++ b/apps/api/src/routes/api/v1/servers/handlers.ts @@ -8,9 +8,7 @@ export async function listServers(request: FastifyRequest) { try { const userId = request.user.userId; const teamId = request.user.teamId; - const servers = await prisma.destinationDocker.findMany({ where: { teams: { some: { id: teamId === '0' ? undefined : teamId } }, remoteEngine: false }, distinct: ['engine'] }) - // const remoteServers = await prisma.destinationDocker.findMany({ where: { teams: { some: { id: teamId === '0' ? undefined : teamId } } }, distinct: ['remoteIpAddress', 'engine'] }) - + const servers = await prisma.destinationDocker.findMany({ where: { teams: { some: { id: teamId === '0' ? undefined : teamId } }}, distinct: ['remoteIpAddress', 'engine'] }) return { servers } @@ -67,8 +65,7 @@ export async function showUsage(request: FastifyRequest) { const { stdout: stats } = await executeSSHCmd({ dockerId: id, command: `vmstat -s` }) const { stdout: disks } = await executeSSHCmd({ dockerId: id, command: `df -m / --output=size,used,pcent|grep -v 'Used'| xargs` }) const { stdout: cpus } = await executeSSHCmd({ dockerId: id, command: `nproc --all` }) - // const { stdout: cpuUsage } = await executeSSHCmd({ dockerId: id, command: `echo $[100-$(vmstat 1 2|tail -1|awk '{print $15}')]` }) - // console.log(cpuUsage) + const { stdout: cpuUsage } = await executeSSHCmd({ dockerId: id, command: `echo $[100-$(vmstat 1 2|tail -1|awk '{print $15}')]` }) const parsed: any = parseFromText(stats) return { usage: { @@ -81,8 +78,8 @@ export async function showUsage(request: FastifyRequest) { freeMemPercentage: (parsed.totalMemoryKB - parsed.usedMemoryKB) / parsed.totalMemoryKB * 100 }, cpu: { - load: 0, - usage: 0, + load: [0,0,0], + usage: cpuUsage, count: cpus }, disk: { diff --git a/apps/api/src/routes/api/v1/services/handlers.ts b/apps/api/src/routes/api/v1/services/handlers.ts index 48e58ca78..2207841d2 100644 --- a/apps/api/src/routes/api/v1/services/handlers.ts +++ b/apps/api/src/routes/api/v1/services/handlers.ts @@ -456,7 +456,7 @@ export async function activatePlausibleUsers(request: FastifyRequest, re if (destinationDockerId) { await executeDockerCmd({ dockerId: destinationDocker.id, - command: `docker exec ${id} 'psql -H postgresql://${postgresqlUser}:${postgresqlPassword}@localhost:5432/${postgresqlDatabase} -c "UPDATE users SET email_verified = true;"'` + command: `docker exec ${id}-postgresql psql -H postgresql://${postgresqlUser}:${postgresqlPassword}@localhost:5432/${postgresqlDatabase} -c "UPDATE users SET email_verified = true;"` }) return await reply.code(201).send() } @@ -476,7 +476,7 @@ export async function cleanupPlausibleLogs(request: FastifyRequest, repl if (destinationDockerId) { await executeDockerCmd({ dockerId: destinationDocker.id, - command: `docker exec ${id}-clickhouse sh -c "/usr/bin/clickhouse-client -q \\"SELECT name FROM system.tables WHERE name LIKE '%log%';\\"| xargs -I{} /usr/bin/clickhouse-client -q \"TRUNCATE TABLE system.{};\""` + command: `docker exec ${id}-clickhouse /usr/bin/clickhouse-client -q \\"SELECT name FROM system.tables WHERE name LIKE '%log%';\\"| xargs -I{} /usr/bin/clickhouse-client -q \"TRUNCATE TABLE system.{};\"` }) return await reply.code(201).send() } diff --git a/apps/api/src/routes/webhooks/github/handlers.ts b/apps/api/src/routes/webhooks/github/handlers.ts index 4bfea692c..5124be0ef 100644 --- a/apps/api/src/routes/webhooks/github/handlers.ts +++ b/apps/api/src/routes/webhooks/github/handlers.ts @@ -1,7 +1,7 @@ import axios from "axios"; import cuid from "cuid"; import crypto from "crypto"; -import { encrypt, errorHandler, getUIUrl, isDev, prisma } from "../../../lib/common"; +import { encrypt, errorHandler, getDomain, getUIUrl, isDev, prisma } from "../../../lib/common"; import { checkContainer, removeContainer } from "../../../lib/docker"; import { createdBranchDatabase, getApplicationFromDBWebhook, removeBranchDatabase } from "../../api/v1/applications/handlers"; @@ -169,10 +169,29 @@ export async function gitHubEvents(request: FastifyRequest): Promi pullmergeRequestAction === 'reopened' || pullmergeRequestAction === 'synchronize' ) { + await prisma.application.update({ where: { id: application.id }, data: { updatedAt: new Date() } }); + let previewApplicationId = undefined + if (pullmergeRequestId) { + const foundPreviewApplications = await prisma.previewApplication.findMany({ where: { applicationId: application.id, pullmergeRequestId } }) + if (foundPreviewApplications.length > 0) { + previewApplicationId = foundPreviewApplications[0].id + } else { + const protocol = application.fqdn.includes('https://') ? 'https://' : 'http://' + const previewApplication = await prisma.previewApplication.create({ + data: { + pullmergeRequestId, + sourceBranch, + customDomain: `${protocol}${pullmergeRequestId}.${getDomain(application.fqdn)}`, + application: { connect: { id: application.id } } + } + }) + previewApplicationId = previewApplication.id + } + } // if (application.connectedDatabase && pullmergeRequestAction === 'opened' || pullmergeRequestAction === 'reopened') { // // Coolify hosted database // if (application.connectedDatabase.databaseId) { @@ -187,6 +206,7 @@ export async function gitHubEvents(request: FastifyRequest): Promi data: { id: buildId, pullmergeRequestId, + previewApplicationId, sourceBranch, applicationId: application.id, destinationDockerId: application.destinationDocker.id, @@ -198,7 +218,9 @@ export async function gitHubEvents(request: FastifyRequest): Promi } }); - + return { + message: 'Queued. Thank you!' + }; } else if (pullmergeRequestAction === 'closed') { if (application.destinationDockerId) { const id = `${application.id}-${pullmergeRequestId}`; @@ -206,13 +228,22 @@ export async function gitHubEvents(request: FastifyRequest): Promi await removeContainer({ id, dockerId: application.destinationDocker.id }); } catch (error) { } } - if (application.connectedDatabase.databaseId) { - const databaseId = application.connectedDatabase.databaseId; - const database = await prisma.database.findUnique({ where: { id: databaseId } }); - if (database) { - await removeBranchDatabase(database, pullmergeRequestId); + const foundPreviewApplications = await prisma.previewApplication.findMany({ where: { applicationId: application.id, pullmergeRequestId } }) + if (foundPreviewApplications.length > 0) { + for (const preview of foundPreviewApplications) { + await prisma.previewApplication.delete({ where: { id: preview.id } }) } } + return { + message: 'PR closed. Thank you!' + }; + // if (application?.connectedDatabase?.databaseId) { + // const databaseId = application.connectedDatabase.databaseId; + // const database = await prisma.database.findUnique({ where: { id: databaseId } }); + // if (database) { + // await removeBranchDatabase(database, pullmergeRequestId); + // } + // } } } } diff --git a/apps/api/src/routes/webhooks/gitlab/handlers.ts b/apps/api/src/routes/webhooks/gitlab/handlers.ts index 264344c4a..8540c5038 100644 --- a/apps/api/src/routes/webhooks/gitlab/handlers.ts +++ b/apps/api/src/routes/webhooks/gitlab/handlers.ts @@ -2,7 +2,7 @@ import axios from "axios"; import cuid from "cuid"; import crypto from "crypto"; import type { FastifyReply, FastifyRequest } from "fastify"; -import { errorHandler, getAPIUrl, getUIUrl, isDev, listSettings, prisma } from "../../../lib/common"; +import { errorHandler, getAPIUrl, getDomain, getUIUrl, isDev, listSettings, prisma } from "../../../lib/common"; import { checkContainer, removeContainer } from "../../../lib/docker"; import { getApplicationFromDB, getApplicationFromDBWebhook } from "../../api/v1/applications/handlers"; @@ -91,8 +91,8 @@ export async function gitLabEvents(request: FastifyRequest) { } } } else if (objectKind === 'merge_request') { - const { object_attributes: { work_in_progress: isDraft, action, source_branch: sourceBranch, target_branch: targetBranch, iid: pullmergeRequestId }, project: { id } } = request.body - + const { object_attributes: { work_in_progress: isDraft, action, source_branch: sourceBranch, target_branch: targetBranch }, project: { id } } = request.body + const pullmergeRequestId = request.body.object_attributes.iid.toString(); const projectId = Number(id); if (!allowedActions.includes(action)) { throw { status: 500, message: 'Action not allowed.' } @@ -130,10 +130,29 @@ export async function gitLabEvents(request: FastifyRequest) { where: { id: application.id }, data: { updatedAt: new Date() } }); + let previewApplicationId = undefined + if (pullmergeRequestId) { + const foundPreviewApplications = await prisma.previewApplication.findMany({ where: { applicationId: application.id, pullmergeRequestId } }) + if (foundPreviewApplications.length > 0) { + previewApplicationId = foundPreviewApplications[0].id + } else { + const protocol = application.fqdn.includes('https://') ? 'https://' : 'http://' + const previewApplication = await prisma.previewApplication.create({ + data: { + pullmergeRequestId, + sourceBranch, + customDomain: `${protocol}${pullmergeRequestId}.${getDomain(application.fqdn)}`, + application: { connect: { id: application.id } } + } + }) + previewApplicationId = previewApplication.id + } + } await prisma.build.create({ data: { id: buildId, - pullmergeRequestId: pullmergeRequestId.toString(), + pullmergeRequestId, + previewApplicationId, sourceBranch, applicationId: application.id, destinationDockerId: application.destinationDocker.id, @@ -150,8 +169,19 @@ export async function gitLabEvents(request: FastifyRequest) { } else if (action === 'close') { if (application.destinationDockerId) { const id = `${application.id}-${pullmergeRequestId}`; - await removeContainer({ id, dockerId: application.destinationDocker.id }); + try { + await removeContainer({ id, dockerId: application.destinationDocker.id }); + } catch (error) { } } + const foundPreviewApplications = await prisma.previewApplication.findMany({ where: { applicationId: application.id, pullmergeRequestId } }) + if (foundPreviewApplications.length > 0) { + for (const preview of foundPreviewApplications) { + await prisma.previewApplication.delete({ where: { id: preview.id } }) + } + } + return { + message: 'MR closed. Thank you!' + }; } } diff --git a/apps/api/src/routes/webhooks/traefik/handlers.ts b/apps/api/src/routes/webhooks/traefik/handlers.ts index 3e2cd5952..d79c2cb01 100644 --- a/apps/api/src/routes/webhooks/traefik/handlers.ts +++ b/apps/api/src/routes/webhooks/traefik/handlers.ts @@ -12,7 +12,7 @@ function configureMiddleware( if (isHttps) { traefik.http.routers[id] = { entrypoints: ['web'], - rule: `Host(\`${nakedDomain}\`) || Host(\`www.${nakedDomain}\`)`, + rule: `(Host(\`${nakedDomain}\`) || Host(\`www.${nakedDomain}\`)) && PathPrefix(\`/\`)`, service: `${id}`, middlewares: ['redirect-to-https'] }; @@ -53,7 +53,7 @@ function configureMiddleware( if (isDualCerts) { traefik.http.routers[`${id}-secure`] = { entrypoints: ['websecure'], - rule: `Host(\`${nakedDomain}\`) || Host(\`www.${nakedDomain}\`)`, + rule: `(Host(\`${nakedDomain}\`) || Host(\`www.${nakedDomain}\`)) && PathPrefix(\`/\`)`, service: `${id}`, tls: { certresolver: 'letsencrypt' @@ -64,7 +64,7 @@ function configureMiddleware( if (isWWW) { traefik.http.routers[`${id}-secure-www`] = { entrypoints: ['websecure'], - rule: `Host(\`www.${nakedDomain}\`)`, + rule: `Host(\`www.${nakedDomain}\`) && PathPrefix(\`/\`)`, service: `${id}`, tls: { certresolver: 'letsencrypt' @@ -73,7 +73,7 @@ function configureMiddleware( }; traefik.http.routers[`${id}-secure`] = { entrypoints: ['websecure'], - rule: `Host(\`${nakedDomain}\`)`, + rule: `Host(\`${nakedDomain}\`) && PathPrefix(\`/\`)`, service: `${id}`, tls: { domains: { @@ -86,7 +86,7 @@ function configureMiddleware( } else { traefik.http.routers[`${id}-secure-www`] = { entrypoints: ['websecure'], - rule: `Host(\`www.${nakedDomain}\`)`, + rule: `Host(\`www.${nakedDomain}\`) && PathPrefix(\`/\`)`, service: `${id}`, tls: { domains: { @@ -97,7 +97,7 @@ function configureMiddleware( }; traefik.http.routers[`${id}-secure`] = { entrypoints: ['websecure'], - rule: `Host(\`${domain}\`)`, + rule: `Host(\`${domain}\`) && PathPrefix(\`/\`)`, service: `${id}`, tls: { certresolver: 'letsencrypt' @@ -110,14 +110,14 @@ function configureMiddleware( } else { traefik.http.routers[id] = { entrypoints: ['web'], - rule: `Host(\`${nakedDomain}\`) || Host(\`www.${nakedDomain}\`)`, + rule: `(Host(\`${nakedDomain}\`) || Host(\`www.${nakedDomain}\`)) && PathPrefix(\`/\`)`, service: `${id}`, middlewares: [] }; traefik.http.routers[`${id}-secure`] = { entrypoints: ['websecure'], - rule: `Host(\`${nakedDomain}\`) || Host(\`www.${nakedDomain}\`)`, + rule: `(Host(\`${nakedDomain}\`) || Host(\`www.${nakedDomain}\`)) && PathPrefix(\`/\`)`, service: `${id}`, tls: { domains: { diff --git a/apps/ui/package.json b/apps/ui/package.json index caf05c233..74e232854 100644 --- a/apps/ui/package.json +++ b/apps/ui/package.json @@ -42,6 +42,7 @@ }, "type": "module", "dependencies": { + "dayjs": "1.11.5", "@sveltejs/adapter-static": "1.0.0-next.39", "@tailwindcss/typography": "^0.5.7", "cuid": "2.1.8", diff --git a/apps/ui/src/lib/common.ts b/apps/ui/src/lib/common.ts index 4af26c100..340fa37d1 100644 --- a/apps/ui/src/lib/common.ts +++ b/apps/ui/src/lib/common.ts @@ -83,4 +83,8 @@ export function handlerNotFoundLoad(error: any, url: URL) { status: 500, error: new Error(`Could not load ${url}`) }; +} + +export function getRndInteger(min: number, max: number) { + return Math.floor(Math.random() * (max - min + 1)) + min; } \ No newline at end of file diff --git a/apps/ui/src/lib/components/Usage.svelte b/apps/ui/src/lib/components/Usage.svelte index 7d4a82929..891aa22be 100644 --- a/apps/ui/src/lib/components/Usage.svelte +++ b/apps/ui/src/lib/components/Usage.svelte @@ -94,9 +94,11 @@ {#if $appSession.teamId === '0'} Cleanup Storage {/if} @@ -108,21 +110,21 @@
Total Memory
-
+
{(usage?.memory?.totalMemMb).toFixed(0)}MB
Used Memory
-
+
{(usage?.memory?.usedMemMb).toFixed(0)}MB
Free Memory
-
+
{(usage?.memory?.freeMemPercentage).toFixed(0)}%
@@ -131,41 +133,41 @@
Total CPU
-
+
{usage?.cpu?.count}
CPU Usage
-
+
{usage?.cpu?.usage}%
Load Average (5,10,30mins)
-
{usage?.cpu?.load}
+
{usage?.cpu?.load}
Total Disk
-
+
{usage?.disk?.totalGb}GB
Used Disk
-
+
{usage?.disk?.usedGb}GB
Free Disk
-
+
{usage?.disk?.freePercentage}%
diff --git a/apps/ui/src/lib/dayjs.ts b/apps/ui/src/lib/dayjs.ts new file mode 100644 index 000000000..9ff5b0a1a --- /dev/null +++ b/apps/ui/src/lib/dayjs.ts @@ -0,0 +1,7 @@ +import dayjs from 'dayjs'; +import utc from 'dayjs/plugin/utc.js'; +import relativeTime from 'dayjs/plugin/relativeTime.js'; +dayjs.extend(utc); +dayjs.extend(relativeTime); + +export { dayjs as day }; diff --git a/apps/ui/src/lib/store.ts b/apps/ui/src/lib/store.ts index b0719e591..19ee6dfe9 100644 --- a/apps/ui/src/lib/store.ts +++ b/apps/ui/src/lib/store.ts @@ -156,4 +156,6 @@ export const addToast = (toast: AddToast) => { let t: any = { ...defaults, ...toast } if (t.timeout) t.timeoutInterval = setTimeout(() => dismissToast(id), t.timeout) toasts.update((all: any) => [t, ...all]) -} \ No newline at end of file +} + +export const selectedBuildId: any = writable(null) \ No newline at end of file diff --git a/apps/ui/src/routes/__layout.svelte b/apps/ui/src/routes/__layout.svelte index 8694dfe45..e1a1b5a4d 100644 --- a/apps/ui/src/routes/__layout.svelte +++ b/apps/ui/src/routes/__layout.svelte @@ -138,7 +138,7 @@ sveltekit:prefetch href="/" class="icons hover:text-white" - class:text-white={$page.url.pathname === '/'} + class:text-pink-500={$page.url.pathname === '/'} class:bg-coolgray-500={$page.url.pathname === '/'} class:bg-coolgray-200={!($page.url.pathname === '/')} > @@ -165,7 +165,7 @@ sveltekit:prefetch href="/servers" class="icons hover:text-white" - class:text-white={$page.url.pathname === '/servers'} + class:text-sky-500={$page.url.pathname === '/servers'} class:bg-coolgray-500={$page.url.pathname === '/servers'} class:bg-coolgray-200={!($page.url.pathname === '/servers')} > @@ -219,6 +219,8 @@ + IAM + + Settings
+ Logout +
@@ -283,12 +290,7 @@ {/if} {/if}
-
+
- -IAM -Settings -Logout diff --git a/apps/ui/src/routes/applications/[id]/_Secret.svelte b/apps/ui/src/routes/applications/[id]/_Secret.svelte index d44c2396e..a35b953a4 100644 --- a/apps/ui/src/routes/applications/[id]/_Secret.svelte +++ b/apps/ui/src/routes/applications/[id]/_Secret.svelte @@ -5,7 +5,6 @@ export let isNewSecret = false; export let isPRMRSecret = false; export let PRMRSecret: any = {}; - if (isPRMRSecret) value = PRMRSecret.value; import { page } from '$app/stores'; @@ -39,7 +38,15 @@ async function createSecret(isNew: any) { try { - if (!name || !value) return; + if (isNew) { + if (!name || !value) return; + } + if (value === undefined && isPRMRSecret) { + return + } + if (value === '' && !isPRMRSecret) { + throw new Error('Value is required.') + } await saveSecret({ isNew, name, @@ -108,7 +115,6 @@ name={isNewSecret ? 'secretValue' : 'secretValueNew'} isPasswordField={true} bind:value - required placeholder="J$#@UIO%HO#$U%H" /> @@ -130,7 +136,7 @@ class:translate-x-0={!isBuildSecret} >
{:else} diff --git a/apps/ui/src/routes/applications/[id]/logs/build.svelte b/apps/ui/src/routes/applications/[id]/logs/build.svelte index de5d60c35..10112edc7 100644 --- a/apps/ui/src/routes/applications/[id]/logs/build.svelte +++ b/apps/ui/src/routes/applications/[id]/logs/build.svelte @@ -23,55 +23,45 @@ export let application: any; export let buildCount: any; import { page } from '$app/stores'; - -import {addToast} from '$lib/store'; + import { addToast, selectedBuildId } from '$lib/store'; import BuildLog from './_BuildLog.svelte'; import { get, post } from '$lib/api'; import { t } from '$lib/translations'; import { changeQueryParams, dateOptions, errorNotification, asyncSleep } from '$lib/common'; import Tooltip from '$lib/components/Tooltip.svelte'; + import { day } from '$lib/dayjs'; + import { onDestroy, onMount } from 'svelte'; + const { id } = $page.params; - let buildId: any; + let loadBuildLogsInterval: any = null; let skip = 0; let noMoreBuilds = buildCount < 5 || buildCount <= skip; - - let buildTook = 0; - const { id } = $page.params; let preselectedBuildId = $page.url.searchParams.get('buildId'); - if (preselectedBuildId) buildId = preselectedBuildId; + if (preselectedBuildId) $selectedBuildId = preselectedBuildId; - async function updateBuildStatus({ detail }: { detail: any }) { - const { status, took } = detail; - if (status !== 'running') { - try { - const data = await get(`/applications/${id}/logs/build?buildId=${buildId}`); - builds = builds.filter((build: any) => { - if (build.id === data.builds[0].id) { - build.status = data.builds[0].status; - build.took = data.builds[0].took; - build.since = data.builds[0].since; - } - return build; - }); - } catch (error) { - return errorNotification(error); - } - } else { - builds = builds.filter((build: any) => { - if (build.id === buildId) build.status = status; - return build; - }); - buildTook = took; - } + onMount(async () => { + getBuildLogs(); + loadBuildLogsInterval = setInterval(() => { + getBuildLogs(); + }, 2000); + + }); + onDestroy(() => { + clearInterval(loadBuildLogsInterval); + }); + async function getBuildLogs() { + const response = await get(`/applications/${$page.params.id}/logs/build?skip=${skip}`); + builds = response.builds; } + async function loadMoreBuilds() { if (buildCount >= skip) { skip = skip + 5; - noMoreBuilds = buildCount >= skip; + noMoreBuilds = buildCount <= skip; try { const data = await get(`/applications/${id}/logs/build?skip=${skip}`); - builds = builds.concat(data.builds); + builds = data.builds return; } catch (error) { return errorNotification(error); @@ -81,26 +71,40 @@ import {addToast} from '$lib/store'; } } function loadBuild(build: any) { - buildId = build; - return changeQueryParams(buildId); + $selectedBuildId = build; + return changeQueryParams($selectedBuildId); } - async function resetQueue() { - const sure = confirm('It will reset all build queues for all applications. If something is queued, it will be canceled automatically. Are you sure? '); + async function resetQueue() { + const sure = confirm( + 'It will reset all build queues for all applications. If something is queued, it will be canceled automatically. Are you sure? ' + ); if (sure) { - - try { - await post(`/internal/resetQueue`, {}); - addToast({ + try { + await post(`/internal/resetQueue`, {}); + addToast({ message: 'Queue reset done.', type: 'success' - }); - await asyncSleep(500) - return window.location.reload() - } catch (error) { - return errorNotification(error); + }); + await asyncSleep(500); + return window.location.reload(); + } catch (error) { + return errorNotification(error); + } } - } - } + } + function generateBadgeColors(status: string) { + if (status === 'failed') { + return 'text-red-500'; + } else if (status === 'running') { + return 'text-yellow-300'; + } else if (status === 'success') { + return 'text-green-500'; + } else if (status === 'canceled') { + return 'text-orange-500'; + } else { + return 'text-white'; + } + }
@@ -156,7 +160,9 @@ import {addToast} from '$lib/store';
- +
{#each builds as build, index (build.id)}
loadBuild(build.id)} class:rounded-tr={index === 0} class:rounded-br={index === builds.length - 1} - class="flex cursor-pointer items-center justify-center py-4 no-underline transition-all duration-100 hover:bg-coolgray-400 hover:shadow-xl" - class:bg-coolgray-400={buildId === build.id} + class="flex cursor-pointer items-center justify-center py-4 no-underline transition-all duration-100 hover:bg-coolgray-300 hover:shadow-xl" + class:bg-coolgray-200={$selectedBuildId === build.id} >
@@ -174,50 +180,55 @@ import {addToast} from '$lib/store';
{build.type}
-
{build.status}
+
+ {build.status} +
{#if build.status === 'running'} -
{$t('application.build.running')}
- Elapsed - {buildTook}s + {build.elapsed}s
- {:else if build.status === 'queued'} -
{$t('application.build.queued')}
- {:else} -
{build.since}
+ {:else if build.status !== 'queued'} +
{day(build.updatedAt).utc().fromNow()}
- {$t('application.build.finished_in')} {build.took}s + {$t('application.build.finished_in')} + {day(build.updatedAt).utc().diff(day(build.createdAt)) / 1000}s
{/if}
{new Intl.DateTimeFormat('default', dateOptions).format(new Date(build.createdAt)) + - `\n${build.status}`} {/each}
{#if !noMoreBuilds} {#if buildCount > 5} -
- +
{/if} {/if}
- {#if buildId} - {#key buildId} - + {#if $selectedBuildId} + {#key $selectedBuildId} + {/key} {/if}
diff --git a/apps/ui/src/routes/applications/[id]/previews.svelte b/apps/ui/src/routes/applications/[id]/previews.svelte deleted file mode 100644 index 2942b2954..000000000 --- a/apps/ui/src/routes/applications/[id]/previews.svelte +++ /dev/null @@ -1,222 +0,0 @@ - - - - -
-
-
- Preview Deployments -
- {application?.name} -
- {#if application.gitSource?.htmlUrl && application.repository && application.branch} - - {#if application.gitSource?.type === 'gitlab'} - - - - {:else if application.gitSource?.type === 'github'} - - - - {/if} - - {/if} -
-{#if loading.init} - -{:else} -
-
- Useful for creating staging environments." - : "These values overwrite application secrets in PR/MR deployments.
Useful for creating staging environments."} - /> -
- {#if applicationSecrets.length !== 0} - - - - - - - - - - - {#each applicationSecrets as secret} - {#key secret.id} - - s.name === secret.name)} - isPRMRSecret - name={secret.name} - value={secret.value} - isBuildSecret={secret.isBuildSecret} - on:refresh={refreshSecrets} - /> - - {/key} - {/each} - -
{$t('forms.name')}{$t('forms.value')}{$t('application.preview.need_during_buildtime')}{$t('forms.action')}
- {/if} -
- -
-
- {#if containers.length > 0} - {#each containers as container} - -
-
{getDomain(container.fqdn)}
-
- -
- -
-
- -
- {/each} - {:else} -
-
- {$t('application.preview.no_previews_available')} -
-
- {/if} -
-
-{/if} diff --git a/apps/ui/src/routes/applications/[id]/previews/index.svelte b/apps/ui/src/routes/applications/[id]/previews/index.svelte new file mode 100644 index 000000000..a46daf99a --- /dev/null +++ b/apps/ui/src/routes/applications/[id]/previews/index.svelte @@ -0,0 +1,436 @@ + + + + +
+
+
+ Preview Deployments +
+ {application?.name} +
+ {#if application.gitSource?.htmlUrl && application.repository && application.branch} + + {#if application.gitSource?.type === 'gitlab'} + + + + {:else if application.gitSource?.type === 'github'} + + + + {/if} + + {/if} +
+{#if loading.init} +
+
Loading...
+
+{:else} +
+
+ Useful for creating staging environments." + : "These values overwrite application secrets in PR/MR deployments.
Useful for creating staging environments."} + /> +
+
+ (Changed previews process flow in v3.10.4)'} + /> + +
+ {#if applicationSecrets.length !== 0} + + + + + + + + + + + {#each applicationSecrets as secret} + {#key secret.id} + + s.name === secret.name)} + isPRMRSecret + name={secret.name} + value={secret.value} + isBuildSecret={secret.isBuildSecret} + on:refresh={refreshSecrets} + /> + + {/key} + {/each} + +
{$t('forms.name')}{$t('forms.value')}{$t('application.preview.need_during_buildtime')}{$t('forms.action')}
+ {/if} +
+ +
+ {#if application.previewApplication.length > 0} +
+ {#each application.previewApplication as preview} +
+
+ {#await getStatus(preview)} + + {:then} + {#if status[preview.id] === 'running'} + + {:else} + + {/if} + {/await} +
+
+

+ PR #{preview.pullmergeRequestId} + {#if status[preview.id] === 'building'} + + BUILDING + + {/if} +

+
+

{preview.customDomain.replace('https://', '').replace('http://', '')}

+
+ +
+ {#if preview.customDomain} + + + + + + + + + {/if} + Open Preview +
+ {#if loading.restart} + + {:else} + + {/if} + + Restart (useful to change secrets) + + Force redeploy (without cache) +
+ + Delete Preview +
+
+
+
+
+ {/each} +
+ {:else} +
+
Previews will shown here.
+
+ {/if} +
+{/if} diff --git a/apps/ui/src/routes/applications/[id]/utils.ts b/apps/ui/src/routes/applications/[id]/utils.ts index d060cdb00..7805024e1 100644 --- a/apps/ui/src/routes/applications/[id]/utils.ts +++ b/apps/ui/src/routes/applications/[id]/utils.ts @@ -22,7 +22,7 @@ export async function saveSecret({ applicationId }: Props): Promise { if (!name) return errorNotification(`${t.get('forms.name')} ${t.get('forms.is_required')}`); - if (!value) return errorNotification(`${t.get('forms.value')} ${t.get('forms.is_required')}`); + if (!value && isNew) return errorNotification(`${t.get('forms.value')} ${t.get('forms.is_required')}`); try { await post(`/applications/${applicationId}/secrets`, { name, diff --git a/apps/ui/src/routes/index.svelte b/apps/ui/src/routes/index.svelte index 673390ffd..731e4ce91 100644 --- a/apps/ui/src/routes/index.svelte +++ b/apps/ui/src/routes/index.svelte @@ -31,7 +31,7 @@ import { get, post } from '$lib/api'; import Usage from '$lib/components/Usage.svelte'; import { t } from '$lib/translations'; - import { asyncSleep } from '$lib/common'; + import { asyncSleep, getRndInteger } from '$lib/common'; import { appSession, search, addToast} from '$lib/store'; import ApplicationsIcons from '$lib/components/svg/applications/ApplicationIcons.svelte'; @@ -87,9 +87,7 @@ filtered.destinations = []; filtered.otherDestinations = []; } - function getRndInteger(min: number, max: number) { - return Math.floor(Math.random() * (max - min + 1)) + min; - } + async function getStatus(resources: any) { const { id, buildPack, dualCerts } = resources; diff --git a/apps/ui/src/routes/login.svelte b/apps/ui/src/routes/login.svelte index 0fe0e6193..ab1799c19 100644 --- a/apps/ui/src/routes/login.svelte +++ b/apps/ui/src/routes/login.svelte @@ -28,11 +28,7 @@ Cookies.set('token', token, { path: '/' }); - $appSession.teamId = payload.teamId; - $appSession.userId = payload.userId; - $appSession.permission = payload.permission; - $appSession.isAdmin = payload.isAdmin; - return await goto('/'); + return window.location.assign('/'); } catch (error) { return errorNotification(error); } finally { diff --git a/apps/ui/src/routes/servers/index.svelte b/apps/ui/src/routes/servers/index.svelte index cb9ef3d17..3be53ef55 100644 --- a/apps/ui/src/routes/servers/index.svelte +++ b/apps/ui/src/routes/servers/index.svelte @@ -37,7 +37,7 @@
{#each servers as server}
-
+
{#if $appSession.teamId === '0'} {/if} @@ -49,4 +49,3 @@

Nothing here.

{/if}
-
Remote servers will be here soon
diff --git a/apps/ui/src/routes/services/[id]/_Services/_PlausibleAnalytics.svelte b/apps/ui/src/routes/services/[id]/_Services/_PlausibleAnalytics.svelte index e1992f56d..fee1d8062 100644 --- a/apps/ui/src/routes/services/[id]/_Services/_PlausibleAnalytics.svelte +++ b/apps/ui/src/routes/services/[id]/_Services/_PlausibleAnalytics.svelte @@ -20,7 +20,9 @@ name="scriptName" id="scriptName" readonly={!$appSession.isAdmin && !$status.service.isRunning} - disabled={!$appSession.isAdmin || $status.service.isRunning} + disabled={!$appSession.isAdmin || + $status.service.isRunning || + $status.service.initialLoading} placeholder="plausible.js" bind:value={service.plausibleAnalytics.scriptName} required @@ -31,7 +33,9 @@ - @type forward - port 24224 - bind 0.0.0.0 - - - - @type http - endpoint http://host.docker.internal:3000/logs.json - - flush_at_shutdown true - flush_mode immediate - flush_thread_count 8 - flush_thread_interval 1 - flush_thread_burst_interval 1 - retry_forever true - retry_type exponential_backoff - - - - - @type parser - key_name log - reserve_data true - - @type json - - \ No newline at end of file diff --git a/others/traefik/docker-compose-tcp.yaml b/others/traefik/docker-compose-tcp.yaml deleted file mode 100644 index 110630d2e..000000000 --- a/others/traefik/docker-compose-tcp.yaml +++ /dev/null @@ -1,23 +0,0 @@ -version: '3.5' - -services: - ${ID}: - container_name: proxy-for-${PORT} - image: traefik:v2.6 - command: - - --api.insecure=true - - --entrypoints.web.address=:${PORT} - - --providers.docker=false - - --providers.docker.exposedbydefault=false - - --providers.http.endpoint=http://host.docker.internal:3000/traefik.json?id=${ID} - - --providers.http.pollTimeout=5s - - --log.level=error - ports: - - '${PORT}:${PORT}' - networks: - - ${NETWORK} - -networks: - net: - external: false - name: ${NETWORK} diff --git a/package.json b/package.json index 96dac690d..8e235b6cf 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "coolify", "description": "An open-source & self-hostable Heroku / Netlify alternative.", - "version": "3.10.3", + "version": "3.10.4", "license": "Apache-2.0", "repository": "github:coollabsio/coolify", "scripts": { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 3e4bb67cb..98d3a1183 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -33,6 +33,8 @@ importers: bree: 9.1.2 cabin: 9.1.2 compare-versions: 5.0.1 + csv-parse: ^5.3.0 + csvtojson: ^2.0.10 cuid: 2.1.8 dayjs: 1.11.5 dockerode: 3.3.4 @@ -65,7 +67,7 @@ importers: typescript: 4.8.2 unique-names-generator: 4.7.1 dependencies: - '@breejs/ts-worker': 2.0.0_d3un4r7p64mpe4ydkpns6lvpxy + '@breejs/ts-worker': 2.0.0_zx7xfusupi724hd5vcuaoj6jni '@fastify/autoload': 5.3.1 '@fastify/cookie': 8.1.0 '@fastify/cors': 8.1.0 @@ -80,6 +82,8 @@ importers: bree: 9.1.2 cabin: 9.1.2 compare-versions: 5.0.1 + csv-parse: 5.3.0 + csvtojson: 2.0.10 cuid: 2.1.8 dayjs: 1.11.5 dockerode: 3.3.4 @@ -144,6 +148,7 @@ importers: classnames: 2.3.1 cuid: 2.1.8 daisyui: 2.24.2 + dayjs: 1.11.5 eslint: 8.23.0 eslint-config-prettier: 8.5.0 eslint-plugin-svelte3: 4.0.0 @@ -169,6 +174,7 @@ importers: '@tailwindcss/typography': 0.5.7_tailwindcss@3.1.8 cuid: 2.1.8 daisyui: 2.24.2_25hquoklqeoqwmt7fwvvcyxm5e + dayjs: 1.11.5 js-cookie: 3.0.1 p-limit: 4.0.0 svelte-select: 4.4.7 @@ -239,11 +245,12 @@ packages: engines: {node: '>= 10'} dev: false - /@breejs/ts-worker/2.0.0_d3un4r7p64mpe4ydkpns6lvpxy: + /@breejs/ts-worker/2.0.0_zx7xfusupi724hd5vcuaoj6jni: resolution: {integrity: sha512-6anHRcmgYlF7mrm/YVRn6rx2cegLuiY3VBxkkimOTWC/dVQeH336imVSuIKEGKTwiuNTPr2hswVdDSneNuXg3A==} engines: {node: '>= 12.11'} peerDependencies: bree: '>=9.0.0' + tsconfig-paths: '>= 4' dependencies: bree: 9.1.2 ts-node: 10.8.2_r4hqq7vrw4pxsipnb7ha25ylfe @@ -1868,6 +1875,10 @@ packages: readable-stream: 3.6.0 dev: false + /bluebird/3.7.2: + resolution: {integrity: sha512-XpNj6GDQzdfW+r2Wnn7xiSAd7TM3jzkxGXBGTtWKuSXv1xUV+azxAm8jdWZN06QTQk+2N2XB9jRDkvbmQmcRtg==} + dev: false + /bn.js/4.12.0: resolution: {integrity: sha512-c98Bf3tPniI+scsdk237ku1Dc3ujXQTSgyiPUDEOe7tRkhrqridvh8klBv0HCEso1OLOYcHuCv/cS6DNxKH+ZA==} dev: false @@ -2290,6 +2301,20 @@ packages: engines: {node: '>=4'} hasBin: true + /csv-parse/5.3.0: + resolution: {integrity: sha512-UXJCGwvJ2fep39purtAn27OUYmxB1JQto+zhZ4QlJpzsirtSFbzLvip1aIgziqNdZp/TptvsKEV5BZSxe10/DQ==} + dev: false + + /csvtojson/2.0.10: + resolution: {integrity: sha512-lUWFxGKyhraKCW8Qghz6Z0f2l/PqB1W3AO0HKJzGIQ5JRSlR651ekJDiGJbBT4sRNNv5ddnSGVEnsxP9XRCVpQ==} + engines: {node: '>=4.0.0'} + hasBin: true + dependencies: + bluebird: 3.7.2 + lodash: 4.17.21 + strip-bom: 2.0.0 + dev: false + /cuid/2.1.8: resolution: {integrity: sha512-xiEMER6E7TlTPnDxrM4eRiC6TRgjNX9xzEZ5U/Se2YJKr7Mq4pJn/2XEHjl3STcSh96GmkHPcBXLES8M29wyyg==} dev: false @@ -3897,6 +3922,10 @@ packages: has-symbols: 1.0.3 dev: true + /is-utf8/0.2.1: + resolution: {integrity: sha512-rMYPYvCzsXywIsldgLaSoPlw5PfoB/ssr7hY4pLfcodrA5M/eArza1a9VmTiNIBNMjOGr1Ow9mTyU2o69U6U9Q==} + dev: false + /is-uuid/1.0.2: resolution: {integrity: sha512-tCByphFcJgf2qmiMo5hMCgNAquNSagOetVetDvBXswGkNfoyEMvGH1yDlF8cbZbKnbVBr4Y5/rlpMz9umxyBkQ==} dev: false @@ -5593,6 +5622,13 @@ packages: ansi-regex: 6.0.1 dev: false + /strip-bom/2.0.0: + resolution: {integrity: sha512-kwrX1y7czp1E69n2ajbG65mIo9dqvJ+8aBQXOGVxqwvNbsXdFM6Lq37dLAY3mknUwru8CfcCbfOLL/gMo+fi3g==} + engines: {node: '>=0.10.0'} + dependencies: + is-utf8: 0.2.1 + dev: false + /strip-bom/3.0.0: resolution: {integrity: sha512-vavAMRXOgBVNF6nyEEmL3DBK19iRpDcoIwW+swQ+CbGiu7lju6t+JklA1MHweoWtadgt4ISVUsXLyDq34ddcwA==} engines: {node: '>=4'}