Sergei Poljanski
0cc3e1dece
datacenterbingo gets a public page showing whether each datacenter's network answers. The probing lives here, next to the schema and the admin, since the public site only has a read-only role. bin/monitor runs as a second container from the same image. Once a minute it opens one TCP connection to every active target (a handshake or a refusal both count as answered) and folds the results into 5-minute rows per datacenter, plus a per-bucket row for control hosts in Russia, so a lost vantage point isn't mistaken for dark sites. Rows are kept 30 days. Targets are managed at /admin/bingo/monitor; only hosts a public source ties to that one site belong there. Migration 017 also grants the new tables to the bingo role, so the public site can read them without a storage redeploy. Assisted-by: Claude:opus-5-5
42 lines
1.4 KiB
Ruby
Executable file
42 lines
1.4 KiB
Ruby
Executable file
#!/usr/bin/env ruby
|
|
# Reachability worker for datacenterbingo's status page (Bingo::Monitor).
|
|
# Runs as its own container from the asxpio image; the web container owns
|
|
# migrations, so this waits until the monitor tables exist.
|
|
require 'logger'
|
|
require 'fileutils'
|
|
|
|
$root = File.expand_path('..', __dir__)
|
|
$logger = Logger.new($stdout)
|
|
$env = ENV.fetch('RACK_ENV', 'development')
|
|
$stdout.sync = true
|
|
|
|
require_relative '../lib/db'
|
|
DB.connect!
|
|
until DB.connection.table_exists?(:bingo_monitor_buckets)
|
|
$logger.info('monitor: waiting for migrations')
|
|
sleep 10
|
|
end
|
|
require_relative '../lib/bingo'
|
|
require_relative '../lib/bingo_monitor'
|
|
|
|
HEARTBEAT = ENV.fetch('MONITOR_HEARTBEAT', '/tmp/monitor.heartbeat')
|
|
pruned_at = nil
|
|
|
|
loop do
|
|
started = Time.now.utc
|
|
begin
|
|
targets = Bingo::Monitor::Target.where(active: true).all
|
|
results = Bingo::Monitor.round(targets)
|
|
Bingo::Monitor.record!(results, at: started)
|
|
if pruned_at.nil? || started - pruned_at > 3600
|
|
Bingo::Monitor.prune!(started)
|
|
pruned_at = started
|
|
end
|
|
FileUtils.touch(HEARTBEAT)
|
|
up = results.count { |_, ms| ms }
|
|
$logger.info("monitor: #{up}/#{results.size} answered in #{(Time.now.utc - started).round(1)}s")
|
|
rescue StandardError => e # a DB blip must not end the worker
|
|
$logger.error("monitor: #{e.class}: #{e.message}")
|
|
end
|
|
sleep [Bingo::Monitor::INTERVAL - (Time.now.utc - started), 1].max
|
|
end
|