{"id":"INC-trigger-931451","kind":"incident","aliases":[],"related":[],"title":"Trigger: us-east-1 region dequeue degradation","description":"Our advice remains the same: switch your default region to us-east-1 if you can. More details below.\n\n**us-east-1**\nOperating normally with global queue times better than normal thresholds. If you have excessive runs queued in us-east-1:\n1. Check your concurrency limits, increase as needed including at the account level from the Concurrency page (let us know via the in-app Help and Feedback and we will credit your account)\n2. If you were on eu-central-1 they could be blocking your us-east-1 runs. Go to the Runs page and filter for \"Queued\" status and \"eu-central-1\" region. Then use the Bulk action on the right to cancel those runs.\n3. When eu-central-1 is back you can replay these runs there.\n\n**eu-central-1**\nThe cluster is being spun back up with more resources and the same settings we used to fix us-east-1. Currently no runs are executing here.\n\nThe same thundering herd issue that happened in us-east-1 caused this issue as everyone moved their traffic over. This cluster was smaller so it took less to cause this problem.\n\n--\n\nAfter this incident is over we will do a full post mortem with what happened along with a long list of things we need to do better. I've been noting them down and we've recorded this entire process.\n\nWe will also be offering credits to all impacted customers. I know this doesn't cover the cost to your businesses.\n\nThis is far from the bar we set for reliability. Sorry everyone,\nMatt","summary":null,"severity":"unknown","cvss":null,"epss":null,"exploited":false,"exploited_at":null,"products":[{"name":"Trigger","fixed":[],"affected":"","ecosystem":"vendor"}],"ecosystems":["vendor"],"tags":["incident","ops","trigger","vendor"],"not_affected_if":[],"how_to_check":null,"fix":null,"status":"open","published_at":"2026-06-22T19:48:00+00:00","first_seen_at":"2026-11-08T16:58:31+00:00","triaged_at":"2026-11-08T16:59:06+00:00","updated_at":"2026-11-09T02:36:08+00:00","category":"vendor","confidence":null,"triage":null,"analyst_status":null,"analyst_note":null,"search":"'1':5A,31C,41C,62C,63C,105C,114C,130C,149C,161C,183C,201C '2':97C '3':144C '931451':12A 'account':74C,96C 'action':136C 'advic':19C 'along':238C 'also':265C 'app':87C 'back':151C,167C 'bar':289C 'better':48C,249C 'block':109C 'bulk':135C 'busi':283C 'cancel':141C 'caus':202C,220C 'central':104C,129C,148C,160C 'check':64C 'cluster':163C,212C 'concurr':66C,78C 'cost':280C 'could':107C 'cover':278C 'credit':94C,268C 'current':184C 'custom':272C 'default':25C 'degrad':8A 'dequeu':7A 'detail':36C 'doesn':276C 'east':4A,30C,40C,61C,113C,182C,200C 'entir':261C 'eu':103C,128C,147C,159C 'eu-centr':102C,127C,146C,158C 'everyon':206C,295C 'excess':55C 'execut':188C 'far':286C 'feedback':90C 'filter':122C 'fix':179C 'full':232C 'global':45C 'go':116C 'happen':196C,237C 'help':88C 'herd':193C 'impact':271C 'in-app':85C 'inc':10A 'inc-trigger':9A 'incid':14B,225C 'includ':71C 'increas':68C 'issu':194C,204C 'know':82C,274C 'less':218C 'let':80C 'level':75C 'limit':67C 'list':242C 'long':241C 'matt':296C 'mortem':234C 'move':207C 'need':70C,246C 'normal':43C,50C 'note':253C 'offer':267C 'op':15B 'oper':42C 'page':79C,120C 'post':233C 'problem':222C 'process':262C 'queu':57C,124C 'queue':46C 'record':259C 'region':6A,26C,131C 'reliabl':293C 'remain':20C 'replay':154C 'resourc':171C 'right':139C 'run':56C,115C,119C,143C,156C,186C 'set':175C,291C 'smaller':214C 'sorri':294C 'spun':166C 'status':125C 'switch':23C 'thing':244C 'threshold':51C 'thunder':192C 'time':47C 'took':217C 'traffic':209C 'trigger':1A,11A,13A,16B 'us':3A,29C,39C,60C,81C,112C,181C,199C 'us-east':2A,28C,38C,59C,111C,180C,198C 'use':133C,177C 've':251C,258C 'vendor':17B 'via':83C","nicknames":[],"mitigations":[],"who_affected":null,"tier":null,"verified":{},"analyst_revisit_at":null,"fix_status":null,"audience":null,"plain_action":null,"pkeys":["trigger"],"ptokens":["trigger"],"labels":["incident","ops","trigger","unknown","vendor"],"campaign_type":null,"search_a":"'1':5A '931451':12A 'degrad':8A 'dequeu':7A 'east':4A 'inc':10A 'inc-trigger':9A 'region':6A 'trigger':1A,11A,13A 'us':3A 'us-east':2A","search_b":"'incid':14B 'op':15B 'trigger':16B 'vendor':17B","search_c":"'1':31C,41C,62C,63C,105C,114C,130C,149C,161C,183C,201C '2':97C '3':144C 'account':74C,96C 'action':136C 'advic':19C 'along':238C 'also':265C 'app':87C 'back':151C,167C 'bar':289C 'better':48C,249C 'block':109C 'bulk':135C 'busi':283C 'cancel':141C 'caus':202C,220C 'central':104C,129C,148C,160C 'check':64C 'cluster':163C,212C 'concurr':66C,78C 'cost':280C 'could':107C 'cover':278C 'credit':94C,268C 'current':184C 'custom':272C 'default':25C 'detail':36C 'doesn':276C 'east':30C,40C,61C,113C,182C,200C 'entir':261C 'eu':103C,128C,147C,159C 'eu-centr':102C,127C,146C,158C 'everyon':206C,295C 'excess':55C 'execut':188C 'far':286C 'feedback':90C 'filter':122C 'fix':179C 'full':232C 'global':45C 'go':116C 'happen':196C,237C 'help':88C 'herd':193C 'impact':271C 'in-app':85C 'incid':225C 'includ':71C 'increas':68C 'issu':194C,204C 'know':82C,274C 'less':218C 'let':80C 'level':75C 'limit':67C 'list':242C 'long':241C 'matt':296C 'mortem':234C 'move':207C 'need':70C,246C 'normal':43C,50C 'note':253C 'offer':267C 'oper':42C 'page':79C,120C 'post':233C 'problem':222C 'process':262C 'queu':57C,124C 'queue':46C 'record':259C 'region':26C,131C 'reliabl':293C 'remain':20C 'replay':154C 'resourc':171C 'right':139C 'run':56C,115C,119C,143C,156C,186C 'set':175C,291C 'smaller':214C 'sorri':294C 'spun':166C 'status':125C 'switch':23C 'thing':244C 'threshold':51C 'thunder':192C 'time':47C 'took':217C 'traffic':209C 'us':29C,39C,60C,81C,112C,181C,199C 'us-east':28C,38C,59C,111C,180C,198C 'use':133C,177C 've':251C,258C 'via':83C","cause_entry":null,"cause_path":null,"disclosures_checked_at":null,"first_public_at":"2026-06-22T19:48:00+00:00","kev":null,"cvss_vector":null,"cvss_by":null,"poc_count":0,"sources":[{"source":"statuspage","external_id":"trigger:931451","url":"https://status.trigger.dev/incident/931451","severity":null,"published_at":"2026-06-22T19:48:00+00:00","modified_at":"2026-06-22T19:48:00+00:00","first_seen_at":"2026-11-08T16:58:31+00:00","fetched_at":"2026-11-09T02:36:08+00:00","note":null}],"documents":[],"disclosures":[],"exploit_sources":[],"notice":"Free to read, with fair use of about 60 pages a minute. For higher limits or API access: sales@infrastack.ai"}