69 Commits

Author SHA1 Message Date
Irfan Paraniya
385affb1d7 feat: Implement cluster metadata key-value store (Closes #302) (#834)
* feat: add cluster metadata key-value store

- Add ClusterMetadata model with ClusterID FK, key, value fields
- Add Metadata []ClusterMetadata relation to Cluster model
- Add CRUD handlers: List, Get, Create, Update, Delete cluster metadata
  - Keys are forced to lowercase on create/update
  - Values preserve case sensitivity
- Add metadata routes under /clusters/{clusterID}/metadata
- Include metadata in ClusterResponse DTO and clusterToDTO mapping
- Add Preload(Metadata) to all cluster queries
- Register ClusterMetadata in AutoMigrate

Closes: internal-GlueOps/issues#302

* feat: include cluster metadata in prepare payload

- Preload cluster Metadata in ClusterPrepareWorker
- Map cluster metadata into mapper.ClusterToDTO response payload

This ensures metadata key-value pairs are injected into the platform JSON payload used by prepare/bootstrap flows.

* feat: add cluster metadata UI section to configure dialog

* feat: simplify cluster metadata to map[string]string in response

* fix: address cluster metadata PR review feedback

Agent-Logs-Url: https://github.com/GlueOps/autoglue/sessions/f767d4b8-ecae-4cde-bb5c-f0845c5a7cdf

Co-authored-by: yesterdaysrebel <256862558+yesterdaysrebel@users.noreply.github.com>

* chore: finalize review feedback updates

Agent-Logs-Url: https://github.com/GlueOps/autoglue/sessions/f767d4b8-ecae-4cde-bb5c-f0845c5a7cdf

Co-authored-by: yesterdaysrebel <256862558+yesterdaysrebel@users.noreply.github.com>

* chore: revert unintended go.sum change

Agent-Logs-Url: https://github.com/GlueOps/autoglue/sessions/f767d4b8-ecae-4cde-bb5c-f0845c5a7cdf

Co-authored-by: yesterdaysrebel <256862558+yesterdaysrebel@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yesterdaysrebel <256862558+yesterdaysrebel@users.noreply.github.com>
2026-04-17 11:05:49 +05:30
Venkat
79a4bfb386 feat: adding more concurrency (#760) 2026-03-07 14:57:53 -08:00
Venkat
20b5b75b02 fix: prevent cluster_action duplicate execution when concurrency > 1 (#759)
Add WithTimeout(168h) to cluster_action registration to prevent
archer's reaper from prematurely reclaiming long-running jobs.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 13:43:42 -08:00
Alanis
c0f51bbee6 Add remote url (#758)
* Add remote url to cluster dto from env

* Delete ui.zip

* Delete Archive.zip

* Delete go.sum

* rebase go sum

* Increase instances from 1 to 10 in cluster action
2026-03-07 19:07:18 +00:00
allanice001
f9e4232b77 stuff 2026-03-05 22:38:09 +00:00
Copilot
060f39de0c Fix concurrent worker race conditions in BastionBootstrapWorker and ClusterActionWorker (#746)
* Initial plan

* fix: prevent apt lock conflicts in concurrent BastionBootstrapWorker instances

Use SELECT FOR UPDATE SKIP LOCKED inside a transaction to atomically
claim pending bastion servers, ensuring each server is processed by
exactly one worker instance even when archer.instances > 1.

Co-authored-by: allanice001 <700853+allanice001@users.noreply.github.com>

* fix: prevent concurrent cluster_action workers from processing the same cluster

Use SELECT FOR UPDATE SKIP LOCKED inside a transaction to atomically
claim a cluster before starting any long-running provisioning work.
Workers that arrive for a cluster already locked (or already in
bootstrapping/provisioning state) fail gracefully with a clear message,
preventing duplicate make-target runs and Docker/apt lock conflicts.

Co-authored-by: allanice001 <700853+allanice001@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: allanice001 <700853+allanice001@users.noreply.github.com>
2026-03-05 21:56:58 +00:00
Alanis
e9e01b41a4 fix: bump WithInstances for cluster_action job (#743) 2026-03-05 19:32:17 +00:00
Alanis
28068191f4 feat: Add remote url (#737)
* Add remote url to cluster dto from env

* Delete ui.zip

* Delete Archive.zip

* Delete go.sum

* rebase go sum
2026-03-04 15:07:13 +00:00
allanice001
5e64edaef0 cleanup 2026-02-28 02:30:45 +00:00
Copilot
80f6435e73 fix: wait for apt/dpkg locks before package installs in bastion bootstrap (#705)
* Initial plan

* fix: add apt lock waiting to bastion bootstrap to prevent dpkg contention failures

On freshly provisioned Ubuntu VMs, background processes like cloud-init
or unattended-upgrades may hold the dpkg lock. This causes apt-get
calls in the bootstrap script to fail with 'Could not get lock
/var/lib/dpkg/lock-frontend'.

Add an apt_wait_lock() helper that polls lock files for up to 5 minutes
before every apt-get invocation (both in pm_update_install and before
the Docker install script).

Co-authored-by: venkatamutyala <6570292+venkatamutyala@users.noreply.github.com>

* fix: address code review - configurable wait timeout and fuser fallback

- Make apt lock wait timeout configurable via APT_LOCK_WAIT_SECS variable
- Add fuser availability check with lsof fallback
- Gracefully skip lock wait if neither fuser nor lsof is available

Co-authored-by: venkatamutyala <6570292+venkatamutyala@users.noreply.github.com>

* Final state - all checks passed

Co-authored-by: venkatamutyala <6570292+venkatamutyala@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: venkatamutyala <6570292+venkatamutyala@users.noreply.github.com>
Co-authored-by: Alanis <allanice001@users.noreply.github.com>
2026-02-27 23:38:56 +00:00
allanice001
0f562ac5f4 ui fixes 2026-01-06 17:05:52 +00:00
allanice001
810218124e revert, but increase httprate 2026-01-06 10:11:42 +00:00
allanice001
5aff256377 fix import issue 2026-01-06 09:59:23 +00:00
Alanis
bdd6b61859 Remove rate limiting by IP from routes
Removed rate limiting middleware from the API routes.
2026-01-06 09:50:10 +00:00
allanice001
79f3259bd6 fix: ensure only a single instance is fired
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-30 17:30:38 +00:00
allanice001
e8d568eba7 fix: ensure only a single instance is fired
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-30 16:35:00 +00:00
allanice001
f8e543b595 fix: improve dns error logging 2025-12-28 00:08:42 +00:00
allanice001
8cc81e52b7 fix: add get record for dns 2025-12-27 23:21:59 +00:00
allanice001
bc3bd92d54 fix: improve job tracking
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-26 15:14:48 +00:00
allanice001
169283b6c7 fix: improve job tracking
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-26 15:04:15 +00:00
allanice001
865270312c fix: update jobs
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-26 04:47:08 +00:00
allanice001
dac28d3ea5 feat: move jobs to action based
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-26 00:30:46 +00:00
allanice001
dd0cefc08a fix: bugfix jobs
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-16 01:15:00 +00:00
allanice001
842f7c9be6 fix: bugfix jobs
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-16 00:52:16 +00:00
allanice001
67d50d2b15 fix: bugfix in responding with correct label ids 2025-12-15 18:04:22 +00:00
allanice001
fd1a81ecd8 fix: api keys form bugfix and org key sweeper job
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-12 01:37:42 +00:00
allanice001
7bef4ef6f1 feat: add org_key and org_secret to payload on bastion
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-12 00:20:27 +00:00
allanice001
21dd26503f feat: add kubeconfig to payload if available
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-11 13:08:39 +00:00
allanice001
5377e521e9 fix: fix build error
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-10 14:19:10 +00:00
allanice001
a929561bc8 fix: bg jobs sequencing 2025-12-10 13:25:12 +00:00
allanice001
4c02179b70 fix: fix payload shape
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-10 12:34:11 +00:00
allanice001
20bef7545c chore: log background jobs
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-09 18:10:50 +00:00
allanice001
fee4c64551 fix: fix background jobs
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-09 15:49:31 +00:00
allanice001
4d37a6363f feat: add docker_image and docker_tag to cluster api
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-08 17:04:10 +00:00
allanice001
45b55015ac fix: ensure jobs are running
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-08 16:18:54 +00:00
allanice001
500a8d1095 chore: ensure build succeeds locally
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-05 12:37:22 +00:00
allanice001
0b342f2c65 fis: updates to remove Terraform Provider reserved word collisions
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-12-05 12:17:36 +00:00
allanice001
efac33fba6 chore: updates in UI due to migration to OAS3.1 2025-11-17 19:57:04 +00:00
allanice001
22a411fed9 chore: updates in UI due to migration to OAS3.1
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-17 19:56:41 +00:00
allanice001
07974c1359 chore: updates in UI due to migration to OAS3.1
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-17 19:56:16 +00:00
allanice001
56f86a11b4 feat: cluster page ui
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-17 18:21:48 +00:00
allanice001
d163a050d8 feat: load balancers ui
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-17 15:16:20 +00:00
allanice001
7985b310c5 feat: Complete AG Loadbalancer & Cluster API
Refactor routing logic (Chi can be a pain when you're managing large sets of routes, but its one of the better options when considering a potential gRPC future)
       Upgrade API Generation to fully support OAS3.1
      Update swagger interface to RapiDoc - the old swagger interface doesnt support OAS3.1 yet
      Docs are now embedded as part of the UI - once logged in they pick up the cookies and org id from what gets set by the UI, but you can override it
      Other updates include better portability of the db-studio

Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-17 04:59:39 +00:00
allanice001
fc1c83ba18 chore: cleanup and route refactoring
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-14 06:12:59 +00:00
allanice001
ad8141a497 feat: adding hourly backups to s3
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-12 05:33:09 +00:00
allanice001
586e51b8cc fix: db-studio prefix fixes 2025-11-11 04:23:27 +00:00
allanice001
ea4c625269 fix: db-studio prefix fixes 2025-11-11 04:01:57 +00:00
allanice001
3a1ce33bca feat: adding embedded db-studio
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-11 03:19:09 +00:00
allanice001
01b48efba0 fix: credentials bugfix
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-10 14:41:16 +00:00
allanice001
8a92727b88 fix: types fixed - credentials
Signed-off-by: allanice001 <allanice001@gmail.com>
2025-11-10 00:02:21 +00:00