From a3caad239cac4695b29769fdfdef77f7c55ecea5 Mon Sep 17 00:00:00 2001 From: peaklabs-dev <122374094+peaklabs-dev@users.noreply.github.com> Date: Wed, 11 Sep 2024 14:09:15 +0200 Subject: [PATCH 1/9] Feat: release.md file --- RELEASE.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 RELEASE.md diff --git a/RELEASE.md b/RELEASE.md new file mode 100644 index 000000000..d362eceaa --- /dev/null +++ b/RELEASE.md @@ -0,0 +1,46 @@ +# Coolify Release Guide + +This guide outlines the release process for Coolify, intended for developers and those interested in understanding how releases are managed and deployed. + +## Release Process + +1. **Development on `next` branch** + - Changes and new features are developed on the `next` or even separate branches. + +2. **Merging to `main`** + - Once changes are ready, they are merged from `next` into the `main` branch. + +3. **Building the release** + - After merging to `main`, a new release is built. + - Note: A push to `main` does not automatically mean a new version is released. + +4. **Creating a GitHub release** + - A new release is created on GitHub, which generates new version images. + - Note: This does not mean the version is released. + +5. **Updating the CDN** + - The final step is updating the version information on the CDN: + [https://cdn.coollabs.io/coolify/versions.json](https://cdn.coollabs.io/coolify/versions.json) + +> [!NOTE] +> The CDN update may not occur immediately after the GitHub release. It can happen hours or even days later due to additional testing, stability checks, or potential hotfixes. + + +## Version Availability + +It's important to understand that a new version released on GitHub may not immediately become available for users to update (through manual or auto-update). + +> [!IMPORTANT] +> If you see a new release on GitHub but haven't received the update, it's likely because the CDN hasn't been updated yet. This is intentional and ensures stability and allows for hotfixes before the new version is officially released. + +### Manually Update to Specific Versions + +> [!WARNING] +> Updating to unreleased versions is not recommended and may cause issues. Use at your own risk! + +To update your Coolify instance to a specific (unreleased) version, use the following command: + +```bash +curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash -s +``` +-> Replace `` with the version you want to update to (for example `4.0.0-beta.332`). \ No newline at end of file From 6498e5b6f01aebac8cf02f3e8a73e35517d75c30 Mon Sep 17 00:00:00 2001 From: peaklabs-dev <122374094+peaklabs-dev@users.noreply.github.com> Date: Wed, 11 Sep 2024 14:16:25 +0200 Subject: [PATCH 2/9] Update RELEASE.md --- RELEASE.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/RELEASE.md b/RELEASE.md index d362eceaa..8c0ff0b41 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -4,8 +4,8 @@ This guide outlines the release process for Coolify, intended for developers and ## Release Process -1. **Development on `next` branch** - - Changes and new features are developed on the `next` or even separate branches. +1. **Development on `next` or separate branches** + - Changes, fixes and new features are developed on the `next` or even separate branches. 2. **Merging to `main`** - Once changes are ready, they are merged from `next` into the `main` branch. @@ -15,8 +15,7 @@ This guide outlines the release process for Coolify, intended for developers and - Note: A push to `main` does not automatically mean a new version is released. 4. **Creating a GitHub release** - - A new release is created on GitHub, which generates new version images. - - Note: This does not mean the version is released. + - A new release is created on GitHub with the new version details. 5. **Updating the CDN** - The final step is updating the version information on the CDN: @@ -35,7 +34,7 @@ It's important to understand that a new version released on GitHub may not immed ### Manually Update to Specific Versions -> [!WARNING] +> [!CAUTION] > Updating to unreleased versions is not recommended and may cause issues. Use at your own risk! To update your Coolify instance to a specific (unreleased) version, use the following command: @@ -43,4 +42,4 @@ To update your Coolify instance to a specific (unreleased) version, use the foll ```bash curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash -s ``` --> Replace `` with the version you want to update to (for example `4.0.0-beta.332`). \ No newline at end of file +-> Replace `` with the version you want to update to (for example `4.0.0-beta.332`). From 3f3bf075fa80b7d313d6b36e49f500242c78d202 Mon Sep 17 00:00:00 2001 From: peaklabs-dev <122374094+peaklabs-dev@users.noreply.github.com> Date: Wed, 11 Sep 2024 14:18:10 +0200 Subject: [PATCH 3/9] Update RELEASE.md --- RELEASE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RELEASE.md b/RELEASE.md index 8c0ff0b41..2cb96b72b 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -32,7 +32,7 @@ It's important to understand that a new version released on GitHub may not immed > [!IMPORTANT] > If you see a new release on GitHub but haven't received the update, it's likely because the CDN hasn't been updated yet. This is intentional and ensures stability and allows for hotfixes before the new version is officially released. -### Manually Update to Specific Versions +## Manually Update to Specific Versions > [!CAUTION] > Updating to unreleased versions is not recommended and may cause issues. Use at your own risk! From 6e48c94c6ea8e06722f4b094533a099f9de1ec0c Mon Sep 17 00:00:00 2001 From: peaklabs-dev <122374094+peaklabs-dev@users.noreply.github.com> Date: Wed, 11 Sep 2024 15:10:25 +0200 Subject: [PATCH 4/9] Install Clockwork --- .env.development.example | 16 +- composer.json | 3 +- composer.lock | 77 ++++++- config/clockwork.php | 424 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 512 insertions(+), 8 deletions(-) create mode 100644 config/clockwork.php diff --git a/.env.development.example b/.env.development.example index ba0213f58..511e3d5fd 100644 --- a/.env.development.example +++ b/.env.development.example @@ -8,12 +8,6 @@ APP_PORT=8000 APP_DEBUG=true MUX_ENABLED=false -# Enable Laravel Telescope for debugging -TELESCOPE_ENABLED=false - -# Selenium Driver URL for Dusk -DUSK_DRIVER_URL=http://selenium:4444 - # PostgreSQL Database Configuration DB_DATABASE=coolify DB_USERNAME=coolify @@ -27,6 +21,16 @@ RAY_ENABLED=false # Set custom ray port RAY_PORT= +# Clockwork Configuration +CLOCKWORK_ENABLED=false +CLOCKWORK_QUEUE_COLLECT=true + +# Enable Laravel Telescope for debugging +TELESCOPE_ENABLED=false + +# Selenium Driver URL for Dusk +DUSK_DRIVER_URL=http://selenium:4444 + # Special Keys for Andras # For cache purging BUNNY_API_KEY= diff --git a/composer.json b/composer.json index 7ca65babe..4dcd066ec 100644 --- a/composer.json +++ b/composer.json @@ -12,6 +12,7 @@ "danharrin/livewire-rate-limiting": "^1.1", "doctrine/dbal": "^3.6", "guzzlehttp/guzzle": "^7.5.0", + "itsgoingd/clockwork": "^5.2", "laravel/fortify": "^v1.16.0", "laravel/framework": "^v11", "laravel/horizon": "^5.27.1", @@ -110,4 +111,4 @@ }, "minimum-stability": "stable", "prefer-stable": true -} \ No newline at end of file +} diff --git a/composer.lock b/composer.lock index 4deaa42de..457a5b7c7 100644 --- a/composer.lock +++ b/composer.lock @@ -4,7 +4,7 @@ "Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies", "This file is @generated automatically" ], - "content-hash": "96f8146407d0e6e897ff097c5eccd3a4", + "content-hash": "5c24125e9c7d34e0d4bc1d1fb1c00018", "packages": [ { "name": "amphp/amp", @@ -2670,6 +2670,81 @@ ], "time": "2023-12-03T19:50:20+00:00" }, + { + "name": "itsgoingd/clockwork", + "version": "v5.2.2", + "source": { + "type": "git", + "url": "https://github.com/itsgoingd/clockwork.git", + "reference": "29bc4cedfbe742b419544c30b7b6e15cd9da08ef" + }, + "dist": { + "type": "zip", + "url": "https://api.github.com/repos/itsgoingd/clockwork/zipball/29bc4cedfbe742b419544c30b7b6e15cd9da08ef", + "reference": "29bc4cedfbe742b419544c30b7b6e15cd9da08ef", + "shasum": "" + }, + "require": { + "ext-json": "*", + "php": ">=5.6" + }, + "suggest": { + "ext-pdo": "Needed in order to use a SQL database for metadata storage", + "ext-pdo_mysql": "Needed in order to use MySQL for metadata storage", + "ext-pdo_postgres": "Needed in order to use Postgres for metadata storage", + "ext-pdo_sqlite": "Needed in order to use a SQLite for metadata storage", + "ext-redis": "Needed in order to use Redis for metadata storage" + }, + "type": "library", + "extra": { + "laravel": { + "providers": [ + "Clockwork\\Support\\Laravel\\ClockworkServiceProvider" + ], + "aliases": { + "Clockwork": "Clockwork\\Support\\Laravel\\Facade" + } + } + }, + "autoload": { + "psr-4": { + "Clockwork\\": "Clockwork/" + } + }, + "notification-url": "https://packagist.org/downloads/", + "license": [ + "MIT" + ], + "authors": [ + { + "name": "itsgoingd", + "email": "itsgoingd@luzer.sk", + "homepage": "https://twitter.com/itsgoingd" + } + ], + "description": "php dev tools in your browser", + "homepage": "https://underground.works/clockwork", + "keywords": [ + "Devtools", + "debugging", + "laravel", + "logging", + "lumen", + "profiling", + "slim" + ], + "support": { + "issues": "https://github.com/itsgoingd/clockwork/issues", + "source": "https://github.com/itsgoingd/clockwork/tree/v5.2.2" + }, + "funding": [ + { + "url": "https://github.com/itsgoingd", + "type": "github" + } + ], + "time": "2024-04-14T10:49:22+00:00" + }, { "name": "jean85/pretty-package-versions", "version": "2.0.6", diff --git a/config/clockwork.php b/config/clockwork.php new file mode 100644 index 000000000..ce880464a --- /dev/null +++ b/config/clockwork.php @@ -0,0 +1,424 @@ + env('CLOCKWORK_ENABLE', null), + + /* + |------------------------------------------------------------------------------------------------------------------ + | Features + |------------------------------------------------------------------------------------------------------------------ + | + | You can enable or disable various Clockwork features here. Some features have additional settings (eg. slow query + | threshold for database queries). + | + */ + + 'features' => [ + + // Cache usage stats and cache queries including results + 'cache' => [ + 'enabled' => env('CLOCKWORK_CACHE_ENABLED', true), + + // Collect cache queries + 'collect_queries' => env('CLOCKWORK_CACHE_QUERIES', true), + + // Collect values from cache queries (high performance impact with a very high number of queries) + 'collect_values' => env('CLOCKWORK_CACHE_COLLECT_VALUES', false) + ], + + // Database usage stats and queries + 'database' => [ + 'enabled' => env('CLOCKWORK_DATABASE_ENABLED', true), + + // Collect database queries (high performance impact with a very high number of queries) + 'collect_queries' => env('CLOCKWORK_DATABASE_COLLECT_QUERIES', true), + + // Collect details of models updates (high performance impact with a lot of model updates) + 'collect_models_actions' => env('CLOCKWORK_DATABASE_COLLECT_MODELS_ACTIONS', true), + + // Collect details of retrieved models (very high performance impact with a lot of models retrieved) + 'collect_models_retrieved' => env('CLOCKWORK_DATABASE_COLLECT_MODELS_RETRIEVED', false), + + // Query execution time threshold in milliseconds after which the query will be marked as slow + 'slow_threshold' => env('CLOCKWORK_DATABASE_SLOW_THRESHOLD'), + + // Collect only slow database queries + 'slow_only' => env('CLOCKWORK_DATABASE_SLOW_ONLY', false), + + // Detect and report duplicate queries + 'detect_duplicate_queries' => env('CLOCKWORK_DATABASE_DETECT_DUPLICATE_QUERIES', false) + ], + + // Dispatched events + 'events' => [ + 'enabled' => env('CLOCKWORK_EVENTS_ENABLED', true), + + // Ignored events (framework events are ignored by default) + 'ignored_events' => [ + // App\Events\UserRegistered::class, + // 'user.registered' + ], + ], + + // Laravel log (you can still log directly to Clockwork with laravel log disabled) + 'log' => [ + 'enabled' => env('CLOCKWORK_LOG_ENABLED', true) + ], + + // Sent notifications + 'notifications' => [ + 'enabled' => env('CLOCKWORK_NOTIFICATIONS_ENABLED', true), + ], + + // Performance metrics + 'performance' => [ + // Allow collecting of client metrics. Requires separate clockwork-browser npm package. + 'client_metrics' => env('CLOCKWORK_PERFORMANCE_CLIENT_METRICS', true) + ], + + // Dispatched queue jobs + 'queue' => [ + 'enabled' => env('CLOCKWORK_QUEUE_ENABLED', true) + ], + + // Redis commands + 'redis' => [ + 'enabled' => env('CLOCKWORK_REDIS_ENABLED', true) + ], + + // Routes list + 'routes' => [ + 'enabled' => env('CLOCKWORK_ROUTES_ENABLED', false), + + // Collect only routes from particular namespaces (only application routes by default) + 'only_namespaces' => [ 'App' ] + ], + + // Rendered views + 'views' => [ + 'enabled' => env('CLOCKWORK_VIEWS_ENABLED', true), + + // Collect views including view data (high performance impact with a high number of views) + 'collect_data' => env('CLOCKWORK_VIEWS_COLLECT_DATA', false), + + // Use Twig profiler instead of Laravel events for apps using laravel-twigbridge (more precise, but does + // not support collecting view data) + 'use_twig_profiler' => env('CLOCKWORK_VIEWS_USE_TWIG_PROFILER', false) + ] + + ], + + /* + |------------------------------------------------------------------------------------------------------------------ + | Enable web UI + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork comes with a web UI accessible via http://your.app/clockwork. Here you can enable or disable this + | feature. You can also set a custom path for the web UI. + | + */ + + 'web' => env('CLOCKWORK_WEB', true), + + /* + |------------------------------------------------------------------------------------------------------------------ + | Enable toolbar + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork can show a toolbar with basic metrics on all responses. Here you can enable or disable this feature. + | Requires a separate clockwork-browser npm library. + | For installation instructions see https://underground.works/clockwork/#docs-viewing-data + | + */ + + 'toolbar' => env('CLOCKWORK_TOOLBAR', true), + + /* + |------------------------------------------------------------------------------------------------------------------ + | HTTP requests collection + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork collects data about HTTP requests to your app. Here you can choose which requests should be collected. + | + */ + + 'requests' => [ + // With on-demand mode enabled, Clockwork will only profile requests when the browser extension is open or you + // manually pass a "clockwork-profile" cookie or get/post data key. + // Optionally you can specify a "secret" that has to be passed as the value to enable profiling. + 'on_demand' => env('CLOCKWORK_REQUESTS_ON_DEMAND', false), + + // Collect only errors (requests with HTTP 4xx and 5xx responses) + 'errors_only' => env('CLOCKWORK_REQUESTS_ERRORS_ONLY', false), + + // Response time threshold in milliseconds after which the request will be marked as slow + 'slow_threshold' => env('CLOCKWORK_REQUESTS_SLOW_THRESHOLD'), + + // Collect only slow requests + 'slow_only' => env('CLOCKWORK_REQUESTS_SLOW_ONLY', false), + + // Sample the collected requests (e.g. set to 100 to collect only 1 in 100 requests) + 'sample' => env('CLOCKWORK_REQUESTS_SAMPLE', false), + + // List of URIs that should not be collected + 'except' => [ + '/horizon/.*', // Laravel Horizon requests + '/telescope/.*', // Laravel Telescope requests + '/_tt/.*', // Laravel Telescope toolbar + '/_debugbar/.*', // Laravel DebugBar requests + ], + + // List of URIs that should be collected, any other URI will not be collected if not empty + 'only' => [ + // '/api/.*' + ], + + // Don't collect OPTIONS requests, mostly used in the CSRF pre-flight requests and are rarely of interest + 'except_preflight' => env('CLOCKWORK_REQUESTS_EXCEPT_PREFLIGHT', true) + ], + + /* + |------------------------------------------------------------------------------------------------------------------ + | Artisan commands collection + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork can collect data about executed artisan commands. Here you can enable and configure which commands + | should be collected. + | + */ + + 'artisan' => [ + // Enable or disable collection of executed Artisan commands + 'collect' => env('CLOCKWORK_ARTISAN_COLLECT', false), + + // List of commands that should not be collected (built-in commands are not collected by default) + 'except' => [ + // 'inspire' + ], + + // List of commands that should be collected, any other command will not be collected if not empty + 'only' => [ + // 'inspire' + ], + + // Enable or disable collection of command output + 'collect_output' => env('CLOCKWORK_ARTISAN_COLLECT_OUTPUT', false), + + // Enable or disable collection of built-in Laravel commands + 'except_laravel_commands' => env('CLOCKWORK_ARTISAN_EXCEPT_LARAVEL_COMMANDS', true) + ], + + /* + |------------------------------------------------------------------------------------------------------------------ + | Queue jobs collection + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork can collect data about executed queue jobs. Here you can enable and configure which queue jobs should + | be collected. + | + */ + + 'queue' => [ + // Enable or disable collection of executed queue jobs + 'collect' => env('CLOCKWORK_QUEUE_COLLECT', false), + + // List of queue jobs that should not be collected + 'except' => [ + // App\Jobs\ExpensiveJob::class + ], + + // List of queue jobs that should be collected, any other queue job will not be collected if not empty + 'only' => [ + // App\Jobs\BuggyJob::class + ] + ], + + /* + |------------------------------------------------------------------------------------------------------------------ + | Tests collection + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork can collect data about executed tests. Here you can enable and configure which tests should be + | collected. + | + */ + + 'tests' => [ + // Enable or disable collection of ran tests + 'collect' => env('CLOCKWORK_TESTS_COLLECT', false), + + // List of tests that should not be collected + 'except' => [ + // Tests\Unit\ExampleTest::class + ] + ], + + /* + |------------------------------------------------------------------------------------------------------------------ + | Enable data collection when Clockwork is disabled + |------------------------------------------------------------------------------------------------------------------ + | + | You can enable this setting to collect data even when Clockwork is disabled, e.g. for future analysis. + | + */ + + 'collect_data_always' => env('CLOCKWORK_COLLECT_DATA_ALWAYS', false), + + /* + |------------------------------------------------------------------------------------------------------------------ + | Metadata storage + |------------------------------------------------------------------------------------------------------------------ + | + | Configure how is the metadata collected by Clockwork stored. Three options are available: + | - files - A simple fast storage implementation storing data in one-per-request files. + | - sql - Stores requests in a sql database. Supports MySQL, PostgreSQL and SQLite. Requires PDO. + | - redis - Stores requests in redis. Requires phpredis. + */ + + 'storage' => env('CLOCKWORK_STORAGE', 'files'), + + // Path where the Clockwork metadata is stored + 'storage_files_path' => env('CLOCKWORK_STORAGE_FILES_PATH', storage_path('clockwork')), + + // Compress the metadata files using gzip, trading a little bit of performance for lower disk usage + 'storage_files_compress' => env('CLOCKWORK_STORAGE_FILES_COMPRESS', false), + + // SQL database to use, can be a name of database configured in database.php or a path to a SQLite file + 'storage_sql_database' => env('CLOCKWORK_STORAGE_SQL_DATABASE', storage_path('clockwork.sqlite')), + + // SQL table name to use, the table is automatically created and updated when needed + 'storage_sql_table' => env('CLOCKWORK_STORAGE_SQL_TABLE', 'clockwork'), + + // Redis connection, name of redis connection or cluster configured in database.php + 'storage_redis' => env('CLOCKWORK_STORAGE_REDIS', 'default'), + + // Redis prefix for Clockwork keys ("clockwork" if not set) + 'storage_redis_prefix' => env('CLOCKWORK_STORAGE_REDIS_PREFIX', 'clockwork'), + + // Maximum lifetime of collected metadata in minutes, older requests will automatically be deleted, false to disable + 'storage_expiration' => env('CLOCKWORK_STORAGE_EXPIRATION', 60 * 24 * 7), + + /* + |------------------------------------------------------------------------------------------------------------------ + | Authentication + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork can be configured to require authentication before allowing access to the collected data. This might be + | useful when the application is publicly accessible. Setting to true will enable a simple authentication with a + | pre-configured password. You can also pass a class name of a custom implementation. + | + */ + + 'authentication' => env('CLOCKWORK_AUTHENTICATION', false), + + // Password for the simple authentication + 'authentication_password' => env('CLOCKWORK_AUTHENTICATION_PASSWORD', 'VerySecretPassword'), + + /* + |------------------------------------------------------------------------------------------------------------------ + | Stack traces collection + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork can collect stack traces for log messages and certain data like database queries. Here you can set + | whether to collect stack traces, limit the number of collected frames and set further configuration. Collecting + | long stack traces considerably increases metadata size. + | + */ + + 'stack_traces' => [ + // Enable or disable collecting of stack traces + 'enabled' => env('CLOCKWORK_STACK_TRACES_ENABLED', true), + + // Limit the number of frames to be collected + 'limit' => env('CLOCKWORK_STACK_TRACES_LIMIT', 10), + + // List of vendor names to skip when determining caller, common vendors are automatically added + 'skip_vendors' => [ + // 'phpunit' + ], + + // List of namespaces to skip when determining caller + 'skip_namespaces' => [ + // 'Laravel' + ], + + // List of class names to skip when determining caller + 'skip_classes' => [ + // App\CustomLog::class + ] + + ], + + /* + |------------------------------------------------------------------------------------------------------------------ + | Serialization + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork serializes the collected data to json for storage and transfer. Here you can configure certain aspects + | of serialization. Serialization has a large effect on the cpu time and memory usage. + | + */ + + // Maximum depth of serialized multi-level arrays and objects + 'serialization_depth' => env('CLOCKWORK_SERIALIZATION_DEPTH', 10), + + // A list of classes that will never be serialized (e.g. a common service container class) + 'serialization_blackbox' => [ + \Illuminate\Container\Container::class, + \Illuminate\Foundation\Application::class, + \Laravel\Lumen\Application::class + ], + + /* + |------------------------------------------------------------------------------------------------------------------ + | Register helpers + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork comes with a "clock" global helper function. You can use this helper to quickly log something and to + | access the Clockwork instance. + | + */ + + 'register_helpers' => env('CLOCKWORK_REGISTER_HELPERS', true), + + /* + |------------------------------------------------------------------------------------------------------------------ + | Send headers for AJAX request + |------------------------------------------------------------------------------------------------------------------ + | + | When trying to collect data, the AJAX method can sometimes fail if it is missing required headers. For example, an + | API might require a version number using Accept headers to route the HTTP request to the correct codebase. + | + */ + + 'headers' => [ + // 'Accept' => 'application/vnd.com.whatever.v1+json', + ], + + /* + |------------------------------------------------------------------------------------------------------------------ + | Server timing + |------------------------------------------------------------------------------------------------------------------ + | + | Clockwork supports the W3C Server Timing specification, which allows for collecting a simple performance metrics + | in a cross-browser way. E.g. in Chrome, your app, database and timeline event timings will be shown in the Dev + | Tools network tab. This setting specifies the max number of timeline events that will be sent. Setting to false + | will disable the feature. + | + */ + + 'server_timing' => env('CLOCKWORK_SERVER_TIMING', 10) + +]; From 25f50603334a03dabb38bf0844b10aebff17c39d Mon Sep 17 00:00:00 2001 From: peaklabs-dev <122374094+peaklabs-dev@users.noreply.github.com> Date: Wed, 11 Sep 2024 15:25:51 +0200 Subject: [PATCH 5/9] move clockwork to dev only --- composer.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/composer.json b/composer.json index 4dcd066ec..9749afdad 100644 --- a/composer.json +++ b/composer.json @@ -12,7 +12,6 @@ "danharrin/livewire-rate-limiting": "^1.1", "doctrine/dbal": "^3.6", "guzzlehttp/guzzle": "^7.5.0", - "itsgoingd/clockwork": "^5.2", "laravel/fortify": "^v1.16.0", "laravel/framework": "^v11", "laravel/horizon": "^5.27.1", @@ -49,6 +48,7 @@ "zircote/swagger-php": "^4.10" }, "require-dev": { + "itsgoingd/clockwork": "^5.2", "fakerphp/faker": "^v1.21.0", "laravel/dusk": "^v8.0", "laravel/pint": "^1.16", From 1a40bebaae0d4b073c569c3a2af9c54ecd97f922 Mon Sep 17 00:00:00 2001 From: Andras Bacsai Date: Wed, 11 Sep 2024 15:47:54 +0200 Subject: [PATCH 6/9] chore: Update permissions in pr-build.yml and version numbers --- .github/workflows/pr-build.yml | 9 +++++++++ config/sentry.php | 2 +- config/version.php | 2 +- versions.json | 6 +++--- 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/.github/workflows/pr-build.yml b/.github/workflows/pr-build.yml index cf2fae8f3..017399e73 100644 --- a/.github/workflows/pr-build.yml +++ b/.github/workflows/pr-build.yml @@ -16,6 +16,11 @@ env: jobs: amd64: runs-on: ubuntu-latest + permissions: + contents: read + packages: write + attestations: write + id-token: write steps: - uses: actions/checkout@v4 - name: Login to ghcr.io @@ -37,6 +42,8 @@ jobs: permissions: contents: read packages: write + attestations: write + id-token: write steps: - uses: actions/checkout@v4 - name: Login to ghcr.io @@ -58,6 +65,8 @@ jobs: permissions: contents: read packages: write + attestations: write + id-token: write needs: [amd64, aarch64] steps: - name: Checkout diff --git a/config/sentry.php b/config/sentry.php index eb0a3a508..5a0bfdaac 100644 --- a/config/sentry.php +++ b/config/sentry.php @@ -7,7 +7,7 @@ return [ // The release version of your application // Example with dynamic git hash: trim(exec('git --git-dir ' . base_path('.git') . ' log --pretty="%h" -n1 HEAD')) - 'release' => '4.0.0-beta.333', + 'release' => '4.0.0-beta.334', // When left empty or `null` the Laravel environment will be used 'environment' => config('app.env'), diff --git a/config/version.php b/config/version.php index c3c14e32b..f4b7ee9f7 100644 --- a/config/version.php +++ b/config/version.php @@ -1,3 +1,3 @@ Date: Wed, 11 Sep 2024 22:44:35 +0300 Subject: [PATCH 7/9] fix: keydb. add `:` delimiter for connection string --- app/Models/StandaloneKeydb.php | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/app/Models/StandaloneKeydb.php b/app/Models/StandaloneKeydb.php index 607cacade..7ecb00348 100644 --- a/app/Models/StandaloneKeydb.php +++ b/app/Models/StandaloneKeydb.php @@ -209,7 +209,7 @@ class StandaloneKeydb extends BaseModel protected function internalDbUrl(): Attribute { return new Attribute( - get: fn () => "redis://{$this->keydb_password}@{$this->uuid}:6379/0", + get: fn () => "redis://:{$this->keydb_password}@{$this->uuid}:6379/0", ); } @@ -218,7 +218,7 @@ class StandaloneKeydb extends BaseModel return new Attribute( get: function () { if ($this->is_public && $this->public_port) { - return "redis://{$this->keydb_password}@{$this->destination->server->getIp}:{$this->public_port}/0"; + return "redis://:{$this->keydb_password}@{$this->destination->server->getIp}:{$this->public_port}/0"; } return null; From 8712af737983a88f7768dd94ecee7e731feff04d Mon Sep 17 00:00:00 2001 From: Andras Bacsai Date: Thu, 12 Sep 2024 09:42:28 +0200 Subject: [PATCH 8/9] fix: move mc command to coolify image from helper --- app/Jobs/DatabaseBackupJob.php | 60 ++++++-------------------------- docker-compose.dev.yml | 1 + docker/coolify-helper/Dockerfile | 2 +- docker/dev/Dockerfile | 3 ++ 4 files changed, 15 insertions(+), 51 deletions(-) diff --git a/app/Jobs/DatabaseBackupJob.php b/app/Jobs/DatabaseBackupJob.php index 5d481199b..e6fa05b55 100644 --- a/app/Jobs/DatabaseBackupJob.php +++ b/app/Jobs/DatabaseBackupJob.php @@ -25,7 +25,7 @@ use Illuminate\Queue\InteractsWithQueue; use Illuminate\Queue\Middleware\WithoutOverlapping; use Illuminate\Queue\SerializesModels; use Illuminate\Support\Str; -use App\Models\InstanceSettings; +use Visus\Cuid2\Cuid2; class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue { @@ -399,6 +399,7 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue $backupCommand .= " $this->container_name pg_dump --format=custom --no-acl --no-owner --username {$this->database->postgres_user} $database > $this->backup_location"; $commands[] = $backupCommand; + ray($commands); $this->backup_output = instant_remote_process($commands, $this->server); $this->backup_output = trim($this->backup_output); if ($this->backup_output === '') { @@ -480,6 +481,7 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue private function upload_to_s3(): void { try { + ray($this->backup_location); if (is_null($this->s3)) { return; } @@ -489,62 +491,20 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue $bucket = $this->s3->bucket; $endpoint = $this->s3->endpoint; $this->s3->testConnection(shouldSave: true); - if (data_get($this->backup, 'database_type') === 'App\Models\ServiceDatabase') { - $network = $this->database->service->destination->network; - } else { - $network = $this->database->destination->network; - } + $configName = new Cuid2; - $this->ensureHelperImageAvailable(); - - $fullImageName = $this->getFullImageName(); - $commands[] = "docker run -d --network {$network} --name backup-of-{$this->backup->uuid} --rm -v $this->backup_location:$this->backup_location:ro {$fullImageName}"; - $commands[] = "docker exec backup-of-{$this->backup->uuid} mc config host add temporary {$endpoint} $key $secret"; - $commands[] = "docker exec backup-of-{$this->backup->uuid} mc cp $this->backup_location temporary/$bucket{$this->backup_dir}/"; + $s3_copy_dir = str($this->backup_location)->replace(backup_dir(), '/var/www/html/storage/app/backups/'); + $commands[] = "docker exec coolify bash -c 'mc config host add {$configName} {$endpoint} $key $secret'"; + $commands[] = "docker exec coolify bash -c 'mc cp $s3_copy_dir {$configName}/{$bucket}{$this->backup_dir}/'"; instant_remote_process($commands, $this->server); $this->add_to_backup_output('Uploaded to S3.'); } catch (\Throwable $e) { $this->add_to_backup_output($e->getMessage()); throw $e; } finally { - $command = "docker rm -f backup-of-{$this->backup->uuid}"; - instant_remote_process([$command], $this->server); + $removeConfigCommands[] = "docker exec coolify bash -c 'mc config remove {$configName}'"; + $removeConfigCommands[] = "docker exec coolify bash -c 'mc alias rm {$configName}'"; + instant_remote_process($removeConfigCommands, $this->server, false); } } - - private function ensureHelperImageAvailable(): void - { - $fullImageName = $this->getFullImageName(); - - $imageExists = $this->checkImageExists($fullImageName); - - if (!$imageExists) { - $this->pullHelperImage($fullImageName); - } - } - - private function checkImageExists(string $fullImageName): bool - { - $result = instant_remote_process(["docker image inspect {$fullImageName} >/dev/null 2>&1 && echo 'exists' || echo 'not exists'"], $this->server, false); - return trim($result) === 'exists'; - } - - private function pullHelperImage(string $fullImageName): void - { - try { - instant_remote_process(["docker pull {$fullImageName}"], $this->server); - } catch (\Exception $e) { - $errorMessage = "Failed to pull helper image: " . $e->getMessage(); - $this->add_to_backup_output($errorMessage); - throw new \RuntimeException($errorMessage); - } - } - - private function getFullImageName(): string - { - $settings = InstanceSettings::get(); - $helperImage = config('coolify.helper_image'); - $latestVersion = $settings->helper_version; - return "{$helperImage}:{$latestVersion}"; - } } diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml index 7eda14d41..718d52d04 100644 --- a/docker-compose.dev.yml +++ b/docker-compose.dev.yml @@ -19,6 +19,7 @@ services: PUSHER_APP_SECRET: "${PUSHER_APP_SECRET:-coolify}" volumes: - .:/var/www/html/:cached + - /data/coolify/backups/:/var/www/html/storage/app/backups postgres: pull_policy: always ports: diff --git a/docker/coolify-helper/Dockerfile b/docker/coolify-helper/Dockerfile index 09ca18825..7aa9d8722 100644 --- a/docker/coolify-helper/Dockerfile +++ b/docker/coolify-helper/Dockerfile @@ -34,7 +34,7 @@ RUN if [[ ${TARGETPLATFORM} == 'linux/arm64' ]]; then \ chmod +x ~/.docker/cli-plugins/docker-compose /usr/bin/docker /usr/local/bin/pack /root/.docker/cli-plugins/docker-buildx \ ;fi -COPY --from=minio/mc:RELEASE.2024-03-13T23-51-57Z /usr/bin/mc /usr/bin/mc +COPY --from=minio/mc:RELEASE.2024-09-09T07-53-10Z /usr/bin/mc /usr/bin/mc RUN chmod +x /usr/bin/mc ENTRYPOINT ["/sbin/tini", "--"] diff --git a/docker/dev/Dockerfile b/docker/dev/Dockerfile index f75a0ff1e..63832dc36 100644 --- a/docker/dev/Dockerfile +++ b/docker/dev/Dockerfile @@ -37,6 +37,9 @@ RUN /bin/bash -c "if [[ ${TARGETPLATFORM} == 'linux/arm64' ]]; then \ curl -L https://github.com/cloudflare/cloudflared/releases/download/${CLOUDFLARED_VERSION}/cloudflared-linux-arm64 -o /usr/local/bin/cloudflared && chmod +x /usr/local/bin/cloudflared \ ;fi" +COPY --from=minio/mc:RELEASE.2024-09-09T07-53-10Z /usr/bin/mc /usr/bin/mc +RUN chmod +x /usr/bin/mc + RUN { \ echo 'upload_max_filesize=256M'; \ echo 'post_max_size=256M'; \ From f672a08afb2dc5833bee63321f6cda545a70ee23 Mon Sep 17 00:00:00 2001 From: Andras Bacsai Date: Thu, 12 Sep 2024 09:56:48 +0200 Subject: [PATCH 9/9] chore: Add minio/mc command to Dockerfile --- docker/prod/Dockerfile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docker/prod/Dockerfile b/docker/prod/Dockerfile index f46124062..d0cebcbca 100644 --- a/docker/prod/Dockerfile +++ b/docker/prod/Dockerfile @@ -68,3 +68,6 @@ RUN { \ echo 'upload_max_filesize=256M'; \ echo 'post_max_size=256M'; \ } > /etc/php/current_version/cli/conf.d/upload-limits.ini + +COPY --from=minio/mc:RELEASE.2024-09-09T07-53-10Z /usr/bin/mc /usr/bin/mc +RUN chmod +x /usr/bin/mc