{"id":"INC-northflank-cmj1i24qr0106uhdob8a4bf2w","kind":"incident","aliases":[],"related":[],"title":"Northflank: Node infrastructure stability - London Region","description":"Type: Incident\n    Duration: 39 minutes\n\n    Affected Components: Networking, Addons, Jobs, Services, \nNorthflank Platform →\n    Dec 11, 14:33:03 GMT+0 - Resolved - The incident has been resolved and the cause identified. A node image security release lead to the host filesystem going into a read only mode when interacting with a specific set of workloads causing the node to become unresponsive. We have implemented a mitigation and are working on a permanent solution. Dec 11, 14:13:28 GMT+0 - Monitoring - The mitigation is in place and the region is operating as expected. Dec 11, 13:54:32 GMT+0 - Identified - There are currently issues with node stability in the London region leading to partial outages for some workloads. The team has identified the issue and is implementing a mitigation.","summary":"A node image security release made the host filesystem go read-only when certain workloads ran, so nodes in Northflank's London region became unresponsive. The incident was a stability outage, and the sources describe no attacker gain.","severity":"unknown","cvss":null,"epss":null,"exploited":false,"exploited_at":null,"products":[{"name":"Northflank","fixed":[],"affected":"","ecosystem":"vendor"}],"ecosystems":["vendor"],"tags":["incident","kubernetes","london-region","northflank","security","vendor"],"not_affected_if":["other-regions"],"how_to_check":null,"fix":null,"status":"resolved","published_at":"2025-12-11T13:54:32+00:00","first_seen_at":"2026-11-08T16:58:28+00:00","triaged_at":"2026-11-08T17:03:24+00:00","updated_at":"2026-11-08T17:03:24+00:00","category":"vendor","confidence":0.5,"triage":{"at":"2026-11-08T17:03:24Z","model":"claude-haiku-5-5","output":{"fix":"unknown","reach":"unknown","title":"Northflank: London region node instability caused partial workload outages","impact":"low","summary":"A node image security release made the host filesystem go read-only when certain workloads ran, so nodes in Northflank's London region became unresponsive. The incident was a stability outage, and the sources describe no attacker gain.","audience":"businesses","category":"vendor","nicknames":[],"who_cares":["northflank","london-region","kubernetes"],"confidence":0.5,"fix_status":"unknown","needs_auth":"unknown","worst_case":"Nodes become unresponsive and some workloads go down","analyst_note":"","how_to_check":"unknown","plain_action":"","product_name":"","needs_analyst":false,"not_affected_if":["other-regions"],"who_is_affected":"Teams running workloads on Northflank's London region may have seen partial outages for some workloads between 13:54 and 14:33 GMT on Dec 11. The sources do not describe a data path or configuration that makes a workload exposed to attack, and they do not say which workloads triggered the read-only filesystem. Teams not using London nodes are not described as affected.","user_interaction":"no","product_ecosystem":"","needs_special_setup":"unknown","severity_if_unknown":"unknown"},"cost_usd":0.000689,"input_tokens":4745,"output_tokens":429,"severity_before":"unknown"},"analyst_status":null,"analyst_note":null,"search":"'+0':78C,137C,157C '03':76C '11':73C,132C,152C '13':134C,153C '14':74C,133C '28':135C '32':155C '33':75C '39':62C '54':154C 'addon':67C 'affect':64C 'attack':57B 'becam':44B 'becom':117C 'caus':87C,113C 'certain':34B 'cmj1i24qr0106uhdob8a4bf2w':10A 'compon':65C 'current':161C 'dec':72C,131C,151C 'describ':55B 'durat':61C 'expect':150C 'filesystem':28B,98C 'gain':58B 'gmt':77C,136C,156C 'go':29B,99C 'host':27B,97C 'identifi':88C,158C,180C 'imag':22B,91C 'implement':121C,185C 'inc':8A 'inc-northflank-cmj1i24qr0106uhdob8a4bf2w':7A 'incid':12B,47B,60C,81C 'infrastructur':3A 'interact':106C 'issu':162C,182C 'job':68C 'kubernet':13B 'lead':94C,170C 'london':5A,15B,42B,168C 'london-region':14B 'made':25B 'minut':63C 'mitig':123C,140C,187C 'mode':104C 'monitor':138C 'network':66C 'node':2A,21B,38B,90C,115C,164C 'northflank':1A,9A,11A,17B,40B,70C 'oper':148C 'outag':51B,173C 'partial':172C 'perman':129C 'place':143C 'platform':71C 'ran':36B 'read':31B,102C 'read-on':30B 'region':6A,16B,43B,146C,169C 'releas':24B,93C 'resolv':79C,84C 'secur':18B,23B,92C 'servic':69C 'set':110C 'solut':130C 'sourc':54B 'specif':109C 'stabil':4A,50B,165C 'team':178C 'type':59C 'unrespons':45B,118C 'vendor':19B 'work':126C 'workload':35B,112C,176C","nicknames":[],"mitigations":[],"who_affected":"Teams running workloads on Northflank's London region may have seen partial outages for some workloads between 13:54 and 14:33 GMT on Dec 11. The sources do not describe a data path or configuration that makes a workload exposed to attack, and they do not say which workloads triggered the read-only filesystem. Teams not using London nodes are not described as affected.","tier":"C","verified":{},"analyst_revisit_at":null,"fix_status":"unknown","audience":"businesses","plain_action":null,"pkeys":["northflank"],"ptokens":["northflank"],"labels":["incident","kubernetes","londonregion","northflank","security","unknown","vendor"],"campaign_type":null,"search_a":"'cmj1i24qr0106uhdob8a4bf2w':10A 'inc':8A 'inc-northflank-cmj1i24qr0106uhdob8a4bf2w':7A 'infrastructur':3A 'london':5A 'node':2A 'northflank':1A,9A,11A 'region':6A 'stabil':4A","search_b":"'attack':57B 'becam':44B 'certain':34B 'describ':55B 'filesystem':28B 'gain':58B 'go':29B 'host':27B 'imag':22B 'incid':12B,47B 'kubernet':13B 'london':15B,42B 'london-region':14B 'made':25B 'node':21B,38B 'northflank':17B,40B 'outag':51B 'ran':36B 'read':31B 'read-on':30B 'region':16B,43B 'releas':24B 'secur':18B,23B 'sourc':54B 'stabil':50B 'unrespons':45B 'vendor':19B 'workload':35B","search_c":"'+0':78C,137C,157C '03':76C '11':73C,132C,152C '13':134C,153C '14':74C,133C '28':135C '32':155C '33':75C '39':62C '54':154C 'addon':67C 'affect':64C 'becom':117C 'caus':87C,113C 'compon':65C 'current':161C 'dec':72C,131C,151C 'durat':61C 'expect':150C 'filesystem':98C 'gmt':77C,136C,156C 'go':99C 'host':97C 'identifi':88C,158C,180C 'imag':91C 'implement':121C,185C 'incid':60C,81C 'interact':106C 'issu':162C,182C 'job':68C 'lead':94C,170C 'london':168C 'minut':63C 'mitig':123C,140C,187C 'mode':104C 'monitor':138C 'network':66C 'node':90C,115C,164C 'northflank':70C 'oper':148C 'outag':173C 'partial':172C 'perman':129C 'place':143C 'platform':71C 'read':102C 'region':146C,169C 'releas':93C 'resolv':79C,84C 'secur':92C 'servic':69C 'set':110C 'solut':130C 'specif':109C 'stabil':165C 'team':178C 'type':59C 'unrespons':118C 'work':126C 'workload':112C,176C","cause_entry":null,"cause_path":null,"disclosures_checked_at":null,"first_public_at":"2025-12-11T13:54:32+00:00","kev":null,"cvss_vector":null,"cvss_by":null,"poc_count":0,"sources":[{"source":"statuspage","external_id":"northflank:cmj1i24qr0106uhdob8a4bf2w","url":"https://status.northflank.com/incident/cmj1i24qr0106uhdob8a4bf2w","severity":null,"published_at":"2025-12-11T13:54:32+00:00","modified_at":"2025-12-11T13:54:32+00:00","first_seen_at":"2026-11-08T16:58:28+00:00","fetched_at":"2026-11-09T02:40:07+00:00","note":null}],"documents":[],"disclosures":[],"exploit_sources":[],"notice":"Free to read, with fair use of about 60 pages a minute. For higher limits or API access: sales@infrastack.ai"}