Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........X............X.......................X............] duration: 1.857621803s FAIL (2025-07-28 00:38:38 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.857657543s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................X.X..........X.....X.X...........X] duration: 30.239590852s FAIL (2025-07-28 00:38:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ess=45.91.116.13:4155 , piece_id=N2IRR7LH27DX75AZIGOQGPW4BDGFI7CMRKA24M2UYKNB2TORR76Q, seg_pos={0 0}, upload=81590]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF Jul 28 00:37:09.814987852 (85h1m19.537550892s) [seg_pos={0 0}, upload=81590]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:37:09.815062484 (85h1m19.537625524s) [upload=81590]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:37:09.815066463 (85h1m19.537629503s) [upload=81590]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:37:10.095114538 (85h1m19.817677578s) [upload=81590]: Done uploading segments... Jul 28 00:37:10.095134416 (85h1m19.817697446s) [upload=81590]: Upload finished. err: <nil> Jul 28 00:38:02.738662065 (85h2m12.461225135s) [upload=81599]: Starting upload Jul 28 00:38:02.738787566 (85h2m12.461350636s) [upload=81599]: Done starting upload Jul 28 00:38:02.738929106 (85h2m12.461492177s) [upload=81599]: Uploading segments... Jul 28 00:38:02.739135141 (85h2m12.461698201s) [upload=81599]: Got next segment. Inline: false Jul 28 00:38:02.739200586 (85h2m12.461763656s) [upload=81599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:38:02.739208401 (85h2m12.461771491s) [upload=81599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:38:03.190604705 (85h2m12.913167776s) [seg_pos={0 0}, upload=81599]: Begin upload segment... Jul 28 00:38:03.190668337 (85h2m12.913231397s) [seg_pos={0 0}, upload=81599]: Done begin upload segment. Jul 28 00:38:03.190709426 (85h2m12.913272516s) [noise=true , address=45.91.116.14:4190 , piece_id=FP6VAMHTXSCQB3MA4HJ7PK2MS6LBZBX36XR4U36RYJSNCHJHZUKA, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.190720948 (85h2m12.913283988s) [noise=true , address=45.91.116.5:4299 , piece_id=LYURUS26P5XVG6XY5MK6BHRRP6INAID5MZRTFZ4ONZLC6GC5D2ZQ, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.190782266 (85h2m12.913345306s) [noise=true , address=104.36.82.14:4223 , piece_id=V5YTIWRKZY2NZJ6AAOFHOO26VE775LP3SL3VAFHJAEK4VQYO6VRA, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.191494367 (85h2m12.914057407s) [noise=true , address=45.91.116.4:4939 , piece_id=Z6EWOXDGN2RS3PIRSRS63PER3QHSKBGO25R6UAMKO44E7LUK2C4A, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.191514155 (85h2m12.914077216s) [noise=true , address=77.90.31.12:4590 , piece_id=TWTFXYQCMKOBTQHDSSH77XU4NIRVGECUOT7EIZPB5BTAR62PMTTQ, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.192155223 (85h2m12.914718303s) [noise=true , address=104.36.82.12:4151 , piece_id=PLWWMA3ROPRSVB4NJVUEW3TBKRQQ4BJYWHM45BV6QEOPPYHKDEYQ, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.192296152 (85h2m12.914859212s) [noise=true , address=77.90.31.15:4698 , piece_id=V5JJIMZDE7PJ4YKMF32EY4BBZ23TXZGSID4RR25732Q4W4VJ256Q, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.192672862 (85h2m12.915235912s) [noise=true , address=45.91.116.10:4082 , piece_id=DOTFN23XOTU7SJL4QZMAQBD53NHZHKPIODSMAJQQTR2LWKAOZYXA, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.192793383 (85h2m12.915356434s) [noise=true , address=45.91.116.7:4010 , piece_id=X43FRYXKATVJO44TE24ITU4QE2NIIAG3PUDWBNQ4RV4ZWEXWMRIA, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.193135357 (85h2m12.915698427s) [noise=true , address=104.36.82.16:4295 , piece_id=CJK6NMD4PFJPGAFKHBIGDLNEDMFCQJA6RTUCP3MV4AXIMXLPABAA, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.193501498 (85h2m12.916064548s) [noise=true , address=104.36.82.11:4115 , piece_id=SLAUZGYHW44M2HNJ5QVRK3DHQDZGGJ6CWN5LSKAPZSICLJC56BKQ, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.193897396 (85h2m12.916460436s) [noise=true , address=77.90.31.11:4555 , piece_id=ZWELWJLZIAWK4DYIQWZT2FUGZTBTKVJ2QMQGLS4YPG62DYRJFSQQ, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.194266802 (85h2m12.916829852s) [noise=true , address=45.91.116.13:4154 , piece_id=OFHZPW64RCX6YQNFHGIKLK2JJ3ETNC3A5RGGJMQTC6DSMWWR3UPA, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.195223863 (85h2m12.917786913s) [noise=true , address=77.90.31.13:4627 , piece_id=MLOE42AP7JESZJDX6QBJQ473MVOML5TULDZJTOB3EIW7YNXSYXMA, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.196233425 (85h2m12.918796465s) [noise=true , address=77.90.31.14:4661 , piece_id=4TKPGW2FXHZYLCYGFGKA3KRXL3SEEKWRH4MLHCREEX6YA6BBW2UA, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:03.196874782 (85h2m12.919437822s) [noise=true , address=45.91.116.12:4226 , piece_id=WYJOXZI4TCTVIOZIWKKOJ7LMQKJ2Q56TM6AHG2E6GPJ5EMCTMMUQ, seg_pos={0 0}, upload=81599]: Uploading piece... Jul 28 00:38:04.827971375 (85h2m14.550534415s) [upload=81599]: Next returned nil segment Jul 28 00:38:04.827982988 (85h2m14.550546018s) [upload=81599]: Waiting for error group managing segments... Jul 28 00:38:05.207136591 (85h2m14.929699631s) [noise=true , address=77.90.31.12:4590 , piece_id=TWTFXYQCMKOBTQHDSSH77XU4NIRVGECUOT7EIZPB5BTAR62PMTTQ, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.235830782 (85h2m14.958393822s) [noise=true , address=45.91.116.4:4939 , piece_id=Z6EWOXDGN2RS3PIRSRS63PER3QHSKBGO25R6UAMKO44E7LUK2C4A, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.281751633 (85h2m15.004314673s) [noise=true , address=45.91.116.12:4226 , piece_id=WYJOXZI4TCTVIOZIWKKOJ7LMQKJ2Q56TM6AHG2E6GPJ5EMCTMMUQ, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.291618069 (85h2m15.014181119s) [noise=true , address=45.91.116.10:4082 , piece_id=DOTFN23XOTU7SJL4QZMAQBD53NHZHKPIODSMAJQQTR2LWKAOZYXA, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.306946297 (85h2m15.029509337s) [noise=true , address=45.91.116.5:4299 , piece_id=LYURUS26P5XVG6XY5MK6BHRRP6INAID5MZRTFZ4ONZLC6GC5D2ZQ, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.311631491 (85h2m15.034194551s) [noise=true , address=77.90.31.14:4661 , piece_id=4TKPGW2FXHZYLCYGFGKA3KRXL3SEEKWRH4MLHCREEX6YA6BBW2UA, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.321245706 (85h2m15.043808746s) [noise=true , address=45.91.116.7:4010 , piece_id=X43FRYXKATVJO44TE24ITU4QE2NIIAG3PUDWBNQ4RV4ZWEXWMRIA, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.353862883 (85h2m15.076425923s) [noise=true , address=77.90.31.13:4627 , piece_id=MLOE42AP7JESZJDX6QBJQ473MVOML5TULDZJTOB3EIW7YNXSYXMA, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.391572863 (85h2m15.114135903s) [noise=true , address=77.90.31.11:4555 , piece_id=ZWELWJLZIAWK4DYIQWZT2FUGZTBTKVJ2QMQGLS4YPG62DYRJFSQQ, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.395874373 (85h2m15.118437413s) [noise=true , address=77.90.31.15:4698 , piece_id=V5JJIMZDE7PJ4YKMF32EY4BBZ23TXZGSID4RR25732Q4W4VJ256Q, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.398440032 (85h2m15.121003072s) [noise=true , address=45.91.116.14:4190 , piece_id=FP6VAMHTXSCQB3MA4HJ7PK2MS6LBZBX36XR4U36RYJSNCHJHZUKA, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:05.407792857 (85h2m15.130355897s) [noise=true , address=45.91.116.13:4154 , piece_id=OFHZPW64RCX6YQNFHGIKLK2JJ3ETNC3A5RGGJMQTC6DSMWWR3UPA, seg_pos={0 0}, upload=81599]: Done uploading piece. err: <nil> Jul 28 00:38:32.090405603 (85h2m41.812968643s) [noise=true , address=104.36.82.14:4223 , piece_id=V5YTIWRKZY2NZJ6AAOFHOO26VE775LP3SL3VAFHJAEK4VQYO6VRA, seg_pos={0 0}, upload=81599]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:42696->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:32.090425331 (85h2m41.812988371s) [noise=true , address=104.36.82.11:4115 , piece_id=SLAUZGYHW44M2HNJ5QVRK3DHQDZGGJ6CWN5LSKAPZSICLJC56BKQ, seg_pos={0 0}, upload=81599]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:32.090458645 (85h2m41.813021685s) [noise=true , address=104.36.82.16:4295 , piece_id=CJK6NMD4PFJPGAFKHBIGDLNEDMFCQJA6RTUCP3MV4AXIMXLPABAA, seg_pos={0 0}, upload=81599]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 188.166.225.170:48192->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:32.090475527 (85h2m41.813038567s) [noise=true , address=104.36.82.12:4151 , piece_id=PLWWMA3ROPRSVB4NJVUEW3TBKRQQ4BJYWHM45BV6QEOPPYHKDEYQ, seg_pos={0 0}, upload=81599]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:34694->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:32.090505685 (85h2m41.813068725s) [seg_pos={0 0}, upload=81599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:38:32.328034839 (85h2m42.050597879s) [upload=81599]: Done uploading segments... Jul 28 00:38:32.328061009 (85h2m42.050624049s) [upload=81599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.239614608s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 30 out of 60 is failed (threshold=3) [...........................X...XXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.231194223s FAIL (2025-07-28 00:38:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 1) ======LOGS====== .51:4063 , piece_id=UFOHDUIFDYZ7YIBNQ35MTCXPLRLBHBOWBDZIVFFANTBILK4IXIUA, seg_pos={0 0}, upload=81593]: Uploading piece... Jul 28 00:37:24.266483250 (85h1m33.98904629s) [noise=true , address=104.36.82.39:4759 , piece_id=BGQTQ5MAAG223G3JQMKAWUQ3VFQUBUM4TRNNBUYDTXR5RWXFAQSQ, seg_pos={0 0}, upload=81593]: Uploading piece... Jul 28 00:37:24.266528907 (85h1m33.989091947s) [noise=true , address=104.36.82.10:4082 , piece_id=S4LLMRMLLLGBKZVAOU64YRTTPOBUY7RJCRE2XDFSYKZQOHCUKS2A, seg_pos={0 0}, upload=81593]: Uploading piece... Jul 28 00:37:24.266734811 (85h1m33.989297851s) [noise=true , address=104.36.82.14:4232 , piece_id=SGO45O3YHRKDLJHMYC6MDK5XGMVLVIITKKVUMUZUGLC3SW6PUBRQ, seg_pos={0 0}, upload=81593]: Uploading piece... Jul 28 00:37:24.266938702 (85h1m33.989501742s) [noise=true , address=104.36.82.46:4507 , piece_id=AZBZNPIEUANURICPT5HHNIBJOONXOYSZKMRTCMWCGOIK5MQ6O25Q, seg_pos={0 0}, upload=81593]: Uploading piece... Jul 28 00:37:24.267146859 (85h1m33.989709899s) [noise=true , address=64.49.13.139:4473 , piece_id=ZSD2KX4MOD32MYM2ZDCNAOCRURMIWC2K6RXIY54CATNRSDT7ALFA, seg_pos={0 0}, upload=81593]: Uploading piece... Jul 28 00:37:25.455809245 (85h1m35.178372285s) [upload=81593]: Next returned nil segment Jul 28 00:37:25.455836717 (85h1m35.178399758s) [upload=81593]: Waiting for error group managing segments... Jul 28 00:37:25.760609097 (85h1m35.483172137s) [noise=true , address=64.49.13.139:4473 , piece_id=ZSD2KX4MOD32MYM2ZDCNAOCRURMIWC2K6RXIY54CATNRSDT7ALFA, seg_pos={0 0}, upload=81593]: Done uploading piece. err: <nil> Jul 28 00:38:23.180165089 (85h2m32.902728129s) [noise=true , address=104.36.82.55:4203 , piece_id=UMCK26FYUQZU44KYNT4YUB7H6FLMYEUJRMH2X6CGOP7PQRXPRLAA, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:12ZewP4Cfpax2814Hqbtfvp4PzcDX455Gv7WwSTSwt1NTMXzYhU, address:104.36.82.55:4203): protocol: write tcp 188.166.225.170:45912->104.36.82.55:4203: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180209904 (85h2m32.902772944s) [noise=true , address=104.36.82.51:4063 , piece_id=UFOHDUIFDYZ7YIBNQ35MTCXPLRLBHBOWBDZIVFFANTBILK4IXIUA, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:12M7QPxHpz7ENUXau3eW71JSTjCqMkyhY4ueY38qRhUULHtvs6D, address:104.36.82.51:4063): protocol: write tcp 188.166.225.170:40210->104.36.82.51:4063: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180250091 (85h2m32.902813141s) [noise=true , address=104.36.82.10:4082 , piece_id=S4LLMRMLLLGBKZVAOU64YRTTPOBUY7RJCRE2XDFSYKZQOHCUKS2A, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:12DKsNCtqxLvpJwLszdLs74V3eb8NH2VAV9V3bngzUczawmNoLK, address:104.36.82.10:4082): protocol: write tcp 188.166.225.170:37834->104.36.82.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180280529 (85h2m32.902843569s) [noise=true , address=104.36.82.14:4232 , piece_id=SGO45O3YHRKDLJHMYC6MDK5XGMVLVIITKKVUMUZUGLC3SW6PUBRQ, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX, address:104.36.82.14:4232): protocol: write tcp 188.166.225.170:44004->104.36.82.14:4232: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180311869 (85h2m32.902874909s) [noise=true , address=104.36.82.46:4507 , piece_id=AZBZNPIEUANURICPT5HHNIBJOONXOYSZKMRTCMWCGOIK5MQ6O25Q, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:14jurWWPpmxwiBS3XmZysEUwugsceYJqVXFj7BbYpx8UFcTR8R, address:104.36.82.46:4507): protocol: write tcp 188.166.225.170:43634->104.36.82.46:4507: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180343349 (85h2m32.902906379s) [noise=true , address=104.36.82.11:4145 , piece_id=DZU7RUV4B6AYCO5BDUWWFWAV7YMQOXO5JZXY2WDG5JZ3LDFVQG4A, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:12sTUH6Y4HBG8ocrPAuuFJiVrcAm1nhpNhJXLW4HaCMdtYmyukp, address:104.36.82.11:4145): protocol: rpc: dial tcp 104.36.82.11:4145: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180404015 (85h2m32.902967055s) [noise=true , address=104.36.82.41:4767 , piece_id=XU55VNOGLQXZ7UG3X3EKQUMZMQI3VNIDHBRBAM7TWC4N5MM35JGQ, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:104.36.82.41:4767): protocol: write tcp 188.166.225.170:41144->104.36.82.41:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180438031 (85h2m32.903001071s) [noise=true , address=104.36.82.39:4759 , piece_id=BGQTQ5MAAG223G3JQMKAWUQ3VFQUBUM4TRNNBUYDTXR5RWXFAQSQ, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:12AEfTC9phpBk8uTbN6k7oDvz6svgwdPH1PaU8jfuuBxhHTRtTR, address:104.36.82.39:4759): protocol: rpc: dial tcp 104.36.82.39:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180513074 (85h2m32.903076114s) [noise=true , address=104.36.82.12:4177 , piece_id=XZAOYDSQRO6RD5LLRPCKYKR46XKVHAXUGDEPLV6QGFNHFYHE5RUA, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:12rFuJ2JyBGWtaASJNnTdHoeKUY1mCfDYfJ74BkxK64U9it8FKF, address:104.36.82.12:4177): protocol: write tcp 188.166.225.170:58916->104.36.82.12:4177: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180563550 (85h2m32.90312659s) [noise=true , address=104.36.82.15:4266 , piece_id=26EINTJPBFTRG7TVE33NCZSEOKSI57UFKNXZSWKXBX4ZGLSBMEBQ, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:1wH8jka9Jw8JLYdGKuit8XoJjVLJRDKJ5ePgeei16eax6mk3j, address:104.36.82.15:4266): protocol: write tcp 188.166.225.170:41424->104.36.82.15:4266: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180604489 (85h2m32.903167529s) [noise=true , address=104.36.82.40:4968 , piece_id=MDCGQ4AV3HB35W3WOZX2LUCGJFRRYPP7PBC3IIISPO45AFEIYFRA, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:1J98jLqn1SVMYjTYxKnNqYbRxSQ5vaUnq2G7XpVngHTaWRJ8oX, address:104.36.82.40:4968): protocol: rpc: dial tcp 104.36.82.40:4968: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180612875 (85h2m32.903175915s) [noise=true , address=104.36.82.54:4166 , piece_id=5KXZZUWKR5S33GYH6TRAJ5CFXJZE75DRTZYB4NE2DHBLT6RQVKKQ, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:1BV7mWGAwKMXzmZBc3qAQFUXYyWtgQC7QEfG54YJgyJ4U3tFiv, address:104.36.82.54:4166): protocol: write tcp 188.166.225.170:49180->104.36.82.54:4166: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180647992 (85h2m32.903211032s) [noise=true , address=104.36.82.14:4245 , piece_id=XG37BA54JEOWOQ4WH52EVPXZRXLB67GYLE73J62YQ6B4SWXQPXYQ, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:1Zu8teNPTXn638grHKwtJqz4UEY7oFYLx1DHvuVP6RZXcatVqX, address:104.36.82.14:4245): protocol: write tcp 188.166.225.170:47460->104.36.82.14:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180660877 (85h2m32.903223917s) [noise=true , address=104.36.82.9:4010 , piece_id=7T6JRA7PS7TEB26DFFINBUQ7UB23HHROMBP6ZJG6UNSOTKQGFJOA, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:19XkukzF8ZCLdHLTYQTYpX5iF4tvT1iM6Zxxs1ZNQ4r5X4ousp, address:104.36.82.9:4010): protocol: write tcp 188.166.225.170:42590->104.36.82.9:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180675254 (85h2m32.903238294s) [noise=true , address=104.36.82.53:4140 , piece_id=YHSEUMNTKKAMG7W3O2IULBIPHPPCOKRNBWIGHUOX3B67TGITO4SQ, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:12Wv1U4bCkzg9Ywx4EKRkWG8tMffjwiVBEZ6tHr8scNHfdcWyrL, address:104.36.82.53:4140): protocol: write tcp 188.166.225.170:58768->104.36.82.53:4140: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180691185 (85h2m32.903254215s) [noise=true , address=104.36.82.20:4455 , piece_id=NKYO3OMVZJH46ICU2S7BZA5AQUNECCQRIAUX75PWQ62L24EFLW3A, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:1X8uFZXHVMycvk6tELmYQyWoAS2sSFmNNWhvQFApvoWXuXFcj4, address:104.36.82.20:4455): protocol: write tcp 188.166.225.170:35466->104.36.82.20:4455: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180717195 (85h2m32.903280225s) [noise=true , address=104.36.82.52:4094 , piece_id=BJTLX2AB5U5Z5QSM7FTGVOPEFYCGOOEJBGAK3OTXHAQQYU4YM2SQ, seg_pos={0 0}, upload=81593]: Done uploading piece. err: ecclient: upload failed (node:1QYpeobqZ5QtLcuNBh9fHhiKxnbEtpiAaagvgWyTZvGbgbFxH7, address:104.36.82.52:4094): protocol: write tcp 188.166.225.170:44258->104.36.82.52:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:38:23.180745699 (85h2m32.903308729s) [seg_pos={0 0}, upload=81593]: Done waiting for segment. successful: 1 optimal: 14 errs: <nil> Jul 28 00:38:23.409718713 (85h2m33.132281753s) [upload=81593]: Done uploading segments... Jul 28 00:38:23.409746346 (85h2m33.132309386s) [upload=81593]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 1) Duration: 1m0.231206426s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...........................X.......X.XX....X....X.......X...] duration: 5.816133798s FAIL (2025-07-28 00:38:15 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.816153004s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........X............X.......................X...........] duration: 1.84981672s FAIL (2025-07-28 00:38:10 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.849834975s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 29 out of 60 is failed (threshold=3) [............................X...XXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.231662629s FAIL (2025-07-28 00:37:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 2) ======LOGS====== =104.36.82.45:4940 , piece_id=Z6AU7C5OD2QWEFEAEJZ244J4UKK7FG7Y6VT7PENN2YXLYJ3YZM2Q, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:23.730719933 (85h0m33.453282993s) [noise=true , address=104.36.82.15:4262 , piece_id=GZNEFZNVOWSGPJ4FEERKNCT7HRVO4YGNRO3LVAVU5UL2BSL3RS3Q, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:23.730859911 (85h0m33.453422941s) [noise=true , address=104.36.82.14:4229 , piece_id=IR2CMHPIWPOJRKACLHUKR6WODY2QYUKF5U2D6MXKMMRM66QECWZQ, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:23.731137944 (85h0m33.453700984s) [noise=true , address=104.36.82.5:4842 , piece_id=KOAK7K7CIZOKNB7BEB6KP5EJJO4RYFQDQSJD3IESBQY6UU3TFO6Q, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:23.731386289 (85h0m33.453949329s) [noise=true , address=104.36.82.54:4157 , piece_id=5JDBZYOG4B446WAE77YNEUNRF2YZRWPHN4QJP5665ATV5S5N7QZA, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:23.731626378 (85h0m33.454189418s) [noise=true , address=104.36.82.43:4863 , piece_id=ZIKPNZPR5I6A6WB5SX62525MUDJGVALO5KOWMUCSDAUME5NEBN2A, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:23.731692855 (85h0m33.454255895s) [noise=true , address=79.127.201.215:4355 , piece_id=4J22OMFGTJORJOBN56GPGTJZ7YFPOXLH4YYWRFJLEQELBEHFUCGA, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:23.731927033 (85h0m33.454490073s) [noise=true , address=104.36.82.18:4399 , piece_id=OD6UMBRKY4JYMODZ2HVKUHZXZJZNQT3CGUS5PVMKSIMZKSCNHDCQ, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:23.732148737 (85h0m33.454711777s) [noise=true , address=104.36.82.53:4155 , piece_id=EOU25V4X4JQOLCO3HH4YY62R2ZA5FV3JLEZZAOFAN6XRUVLG6E4Q, seg_pos={0 0}, upload=81576]: Uploading piece... Jul 28 00:36:24.892142421 (85h0m34.614705461s) [upload=81576]: Next returned nil segment Jul 28 00:36:24.892164061 (85h0m34.614727101s) [upload=81576]: Waiting for error group managing segments... Jul 28 00:36:25.189753289 (85h0m34.912316339s) [noise=true , address=64.49.13.154:4932 , piece_id=M73RZ3CUUMCODQI2GLDPXRYQPXSGXNZNSUUQ3276DCQDAECJQ7FQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: <nil> Jul 28 00:36:29.598541183 (85h0m39.321104223s) [noise=true , address=79.127.201.215:4355 , piece_id=4J22OMFGTJORJOBN56GPGTJZ7YFPOXLH4YYWRFJLEQELBEHFUCGA, seg_pos={0 0}, upload=81576]: Done uploading piece. err: <nil> Jul 28 00:37:22.639412287 (85h1m32.361975327s) [noise=true , address=104.36.82.15:4262 , piece_id=GZNEFZNVOWSGPJ4FEERKNCT7HRVO4YGNRO3LVAVU5UL2BSL3RS3Q, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:12nPw17QDTyrgDBBNKuunFWuDqPV5Rd2GpM7Loc6Ws9vAAZz22s, address:104.36.82.15:4262): protocol: rpc: dial tcp 104.36.82.15:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639480808 (85h1m32.362043848s) [noise=true , address=104.36.82.12:4168 , piece_id=W6UT7MIYXRARWEVAYBGCL55LX7VTLHOESWUJLNGOCTPQXD6OZ74A, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:12XJ5kTKY8pLq3uDJZfFVkTgS56Xx8pG2R5i2A9VRNDEckdKnTF, address:104.36.82.12:4168): protocol: rpc: dial tcp 104.36.82.12:4168: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639535122 (85h1m32.362098162s) [noise=true , address=104.36.82.42:4830 , piece_id=6DHOI5O3TYJLPCMCLIHTY7JYQFP6PZB7GBHAWDTHR27FV33MTHWQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9, address:104.36.82.42:4830): protocol: rpc: dial tcp 104.36.82.42:4830: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639586661 (85h1m32.362149701s) [noise=true , address=104.36.82.13:4210 , piece_id=LKOFXIAU4YNQALXHRJM55JJI2P4NJ6Y4LOQXKMWKQEVVX4LXHYJQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:1ybrcx8SPgyEsJWq2P1KYaTwTj5v5BfENKNEZZxaKRvKvuqTiL, address:104.36.82.13:4210): protocol: rpc: dial tcp 104.36.82.13:4210: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639635715 (85h1m32.362198755s) [noise=true , address=104.36.82.43:4863 , piece_id=ZIKPNZPR5I6A6WB5SX62525MUDJGVALO5KOWMUCSDAUME5NEBN2A, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:1hoK1e9x7ms16dWqXpNpfHA97U74iX9DkT7n73YUTcePgbMatZ, address:104.36.82.43:4863): protocol: rpc: dial tcp 104.36.82.43:4863: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639690119 (85h1m32.362253159s) [noise=true , address=104.36.82.18:4399 , piece_id=OD6UMBRKY4JYMODZ2HVKUHZXZJZNQT3CGUS5PVMKSIMZKSCNHDCQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:1QPwXaFGpwMLXMiSnkE1eycqZkUSnDYMj9sFYZ2BPrPbenw9uK, address:104.36.82.18:4399): protocol: rpc: dial tcp 104.36.82.18:4399: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639739843 (85h1m32.362302883s) [noise=true , address=104.36.82.53:4155 , piece_id=EOU25V4X4JQOLCO3HH4YY62R2ZA5FV3JLEZZAOFAN6XRUVLG6E4Q, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:17wRBzmgMvpNyXy9SZEwEjwc4W5TfKzTj8vaa9c2HXnPNLMMYi, address:104.36.82.53:4155): protocol: rpc: dial tcp 104.36.82.53:4155: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639808355 (85h1m32.362371395s) [noise=true , address=104.36.82.14:4229 , piece_id=IR2CMHPIWPOJRKACLHUKR6WODY2QYUKF5U2D6MXKMMRM66QECWZQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:12uiCMvJ7pnwXfN57CVu8zR73tikwUZiXr3sG7Xotd9w6KVnViL, address:104.36.82.14:4229): protocol: write tcp 188.166.225.170:37942->104.36.82.14:4229: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639827492 (85h1m32.362390532s) [noise=true , address=104.36.82.20:4442 , piece_id=CXAQEIOMH26XTHKGPRGFTSFHJAJF4NE6AV7W7GWC3JTQP4347AMQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:125KyoqeSzbjP8XPhXqw2AenTDiLtVmt5KpLBD7ZW7YC9e9zJu9, address:104.36.82.20:4442): protocol: rpc: dial tcp 104.36.82.20:4442: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639867628 (85h1m32.362430678s) [noise=true , address=104.36.82.54:4157 , piece_id=5JDBZYOG4B446WAE77YNEUNRF2YZRWPHN4QJP5665ATV5S5N7QZA, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz, address:104.36.82.54:4157): protocol: rpc: dial tcp 104.36.82.54:4157: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639886304 (85h1m32.362449354s) [noise=true , address=104.36.82.41:4776 , piece_id=SC4PNYVR3F3HT5XD7L76TUFMSKHP6WLVZAFL4PLHYDLZ67HHSZMA, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:1Hjw4yBSJfpaGqUWqAqfqc1P9ThM6kwd5vMgbLCrSR8z9eiqH7, address:104.36.82.41:4776): protocol: write tcp 188.166.225.170:36708->104.36.82.41:4776: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639919868 (85h1m32.362482918s) [noise=true , address=104.36.82.5:4842 , piece_id=KOAK7K7CIZOKNB7BEB6KP5EJJO4RYFQDQSJD3IESBQY6UU3TFO6Q, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:12q4VdT5BcjUb16PBYHK6zrnW8dqENKrcGx9zdLtNefDVB1MUSn, address:104.36.82.5:4842): protocol: rpc: dial tcp 104.36.82.5:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639959083 (85h1m32.362522123s) [noise=true , address=104.36.82.56:4261 , piece_id=FPHK2HCZQJEYWC552A6R4TW37QXCKZ2ZUZJIGWCE7WRIM42CLFNQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:12YNKaVEsXzHe6XGQuv6wR1qqbv6dbzg1xT5Aqkew2ZV3z3tMMf, address:104.36.82.56:4261): protocol: rpc: dial tcp 104.36.82.56:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.639972098 (85h1m32.362535138s) [noise=true , address=104.36.82.10:4107 , piece_id=APQSGUREN5MSCDKLOH7EH72AFLCDSPZRINWJXUXM6XQV4NLFQTQQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:104.36.82.10:4107): protocol: write tcp 188.166.225.170:36482->104.36.82.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.640013266 (85h1m32.362576306s) [noise=true , address=104.36.82.45:4940 , piece_id=Z6AU7C5OD2QWEFEAEJZ244J4UKK7FG7Y6VT7PENN2YXLYJ3YZM2Q, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:1ajfrKxYoBvXFurqvTkLDkiY39pAWGgmsMXfdBMVgDKvuhWRdr, address:104.36.82.45:4940): protocol: rpc: dial tcp 104.36.82.45:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.640041711 (85h1m32.362604751s) [noise=true , address=104.36.82.22:4530 , piece_id=EFIMSLM3BTEF3ZELMANKFT3USJRMYEDAIKQ7CFDAD2YXD4JL3IKQ, seg_pos={0 0}, upload=81576]: Done uploading piece. err: ecclient: upload failed (node:1akHUSoQFafzkGTJ385iYaowc87pJBDNfU5hodSCUDFa5ojN3v, address:104.36.82.22:4530): protocol: write tcp 188.166.225.170:56556->104.36.82.22:4530: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:37:22.640089823 (85h1m32.362652853s) [seg_pos={0 0}, upload=81576]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil> Jul 28 00:37:22.868604711 (85h1m32.591167751s) [upload=81576]: Done uploading segments... Jul 28 00:37:22.868631402 (85h1m32.591194442s) [upload=81576]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2) Duration: 1m0.231675755s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........................X.X..........X.....X.X...........] duration: 5.972317602s FAIL (2025-07-28 00:37:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.972343611s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [............................X.......X.XX....X....X.......X..] duration: 5.763701308s FAIL (2025-07-28 00:37:05 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.763729692s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X............X.......................X..........] duration: 1.857680335s FAIL (2025-07-28 00:36:56 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.857709732s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.............................X.......X.XX....X....X.......X.] duration: 5.831861438s FAIL (2025-07-28 00:36:33 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.831888931s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [.............................X...XXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.234720931s FAIL (2025-07-28 00:36:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 1) ======LOGS====== piece... Jul 28 00:35:23.269352912 (84h59m32.991915952s) [noise=true , address=104.36.82.15:4277 , piece_id=6LWJIDNO5URKOPXM3JLQA4EXLOK67OZJTFLTYMXTMVBZZYGYQZDQ, seg_pos={0 0}, upload=81562]: Uploading piece... Jul 28 00:35:23.269466118 (84h59m32.992029158s) [noise=true , address=104.36.82.14:4230 , piece_id=ABQ7QRVKKCPXQWSCQPYHJMZ3KY4ULO2DONZH36HHJTTGZZJWYHCQ, seg_pos={0 0}, upload=81562]: Uploading piece... Jul 28 00:35:23.269612368 (84h59m32.992175398s) [noise=true , address=104.36.82.22:4534 , piece_id=5RT7VVHISZAINBGDXTAOLBCWGGDTZWAU2PRPKJSW5AQGLRZG5W2Q, seg_pos={0 0}, upload=81562]: Uploading piece... Jul 28 00:35:23.269729132 (84h59m32.992292162s) [noise=true , address=104.36.82.55:4212 , piece_id=ECF7CNXJXNVHQM4V6GOTIHNUBH6GW3MNBT5B54Q4EALKHYH7TAJQ, seg_pos={0 0}, upload=81562]: Uploading piece... Jul 28 00:35:23.270026290 (84h59m32.99258932s) [noise=true , address=104.36.82.4:4823 , piece_id=27LYFLYI25EQWWT3AR3IF6V62ETPUL76VHC6R2UNNAMUJ27HVRWQ, seg_pos={0 0}, upload=81562]: Uploading piece... Jul 28 00:35:24.786129718 (84h59m34.508692758s) [upload=81562]: Next returned nil segment Jul 28 00:35:24.786147271 (84h59m34.508710302s) [upload=81562]: Waiting for error group managing segments... Jul 28 00:35:28.159006254 (84h59m37.881569304s) [noise=true , address=79.127.201.215:4358 , piece_id=WCQGOHD4IBEY62YJR3UAEOXMRLU4AEKOIKZKWXRCBZA44DOAEHIQ, seg_pos={0 0}, upload=81562]: Done uploading piece. err: <nil> Jul 28 00:36:22.168242852 (85h0m31.890805892s) [noise=true , address=104.36.82.10:4110 , piece_id=B4RFMNE4W27RCXIM76MSZWLOQF4BEYG4THZCR5CULPSFYC3O4CTA, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi, address:104.36.82.10:4110): protocol: write tcp 188.166.225.170:58348->104.36.82.10:4110: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168289011 (85h0m31.890852041s) [noise=true , address=104.36.82.56:4247 , piece_id=3DVXN6QMQRORBODD623GKFY53K53HWDYR5GWTEJONDGF7GRTGJSQ, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:12PzKreuPT9X594ZTTYbXanNdaFVVJsnKL99fumjXymnRa9jeEq, address:104.36.82.56:4247): protocol: write tcp 188.166.225.170:60228->104.36.82.56:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168325089 (85h0m31.890888119s) [noise=true , address=104.36.82.5:4865 , piece_id=LEA6LQVCH42FAIWSDDCLUBZL4XNXBHVUKEAWKALHRHYXDVLB2YXA, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1kqNGahsBw6VsV3JTALvPyGshuSFaAdRSo94cQnCXF3ak54KtL, address:104.36.82.5:4865): protocol: write tcp 188.166.225.170:58972->104.36.82.5:4865: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168329919 (85h0m31.890892959s) [noise=true , address=104.36.82.4:4823 , piece_id=27LYFLYI25EQWWT3AR3IF6V62ETPUL76VHC6R2UNNAMUJ27HVRWQ, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1JWhB5QxbZsTD863tEfkEN1CUbKBMQvMJ9ZaiMAwKvxxXyDkEq, address:104.36.82.4:4823): protocol: write tcp 188.166.225.170:52362->104.36.82.4:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168382429 (85h0m31.890945469s) [noise=true , address=104.36.82.44:4896 , piece_id=MUFEHJUJRAJOTSXVDII2EWE64SF7ETU2LFMQCQCX54GHLELKCZBA, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:12ndSgjgfZMVQni5tw1ugdTrhJt4c4NagdvgYTbSbjatYYtfu1o, address:104.36.82.44:4896): protocol: write tcp 188.166.225.170:49016->104.36.82.44:4896: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168411515 (85h0m31.890974555s) [noise=true , address=104.36.82.3:4795 , piece_id=UOX7MD5SFADWST5GXA7P43PRH72KFAQ6WH43JSYLQ6HV52JUA7AA, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1rL9dLWui62A3vRgBSNw7QQf7g2uubwW85to4ioCrhyao2Pupo, address:104.36.82.3:4795): protocol: write tcp 188.166.225.170:37886->104.36.82.3:4795: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168419149 (85h0m31.890982189s) [noise=true , address=104.36.82.21:4501 , piece_id=OOSVRQJLBBIU7VLWIYUPEAJTVG6UYVVPFLGB4MJC55XDCYN73UBA, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1e1R3gjh5CYJdDqck7xcR6A19Hb443FWbX8X2AQikxt5f4AVvU, address:104.36.82.21:4501): protocol: write tcp 188.166.225.170:46752->104.36.82.21:4501: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168451953 (85h0m31.891014993s) [noise=true , address=104.36.82.54:4164 , piece_id=LAUJBQOHIUHQMNR6ORTBSNQWRDRAUZZQO5D6ZWLPVIQ677T2DNTQ, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1gUXWffNccBoXWftiMWHHSTeEpA5zrd2cHz4uXX5j75GeEK9s2, address:104.36.82.54:4164): protocol: write tcp 188.166.225.170:33612->104.36.82.54:4164: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168479215 (85h0m31.891042255s) [noise=true , address=104.36.82.5:4861 , piece_id=IQSNXYRYOB5MPF75DGVYDND2UW27FGFEFWZCF33RHZ4XGBLSN6KA, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 188.166.225.170:57942->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168501897 (85h0m31.891064937s) [noise=true , address=104.36.82.53:4142 , piece_id=P2AL43Z2P4M252YI4IZRKPMZ6SJKPAD2MOSYDT72PMBNT7SW4WEA, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1yLPWJs1RfZgFisHUpW3wwiFsvV98yz7adKtAKAK3unq1KZCSK, address:104.36.82.53:4142): protocol: write tcp 188.166.225.170:43384->104.36.82.53:4142: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168517698 (85h0m31.891080738s) [noise=true , address=104.36.82.14:4230 , piece_id=ABQ7QRVKKCPXQWSCQPYHJMZ3KY4ULO2DONZH36HHJTTGZZJWYHCQ, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:15hXnJMgf7FAJJh5y1S35eNLdvvqQR4yFot9M8FAmwJDyYVsWf, address:104.36.82.14:4230): protocol: write tcp 188.166.225.170:34776->104.36.82.14:4230: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168537256 (85h0m31.891100296s) [noise=true , address=104.36.82.22:4534 , piece_id=5RT7VVHISZAINBGDXTAOLBCWGGDTZWAU2PRPKJSW5AQGLRZG5W2Q, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1Y7PoZ6DqyK6p456p8gQW3DRcK3g5Y6s3i55XoQwcMuvgBFd6t, address:104.36.82.22:4534): protocol: write tcp 188.166.225.170:41252->104.36.82.22:4534: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168594124 (85h0m31.891157164s) [noise=true , address=104.36.82.13:4204 , piece_id=ST3LCMJEH4N3Q2LFE5ZR7EHFCBHLA6BALZQLKDCRHCYJYOOPFM3A, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:12R53RRLYuJc7r9Z1W9n6guzrr5wF4T4vuGMEzjWAPBJwSCJq1g, address:104.36.82.13:4204): protocol: write tcp 188.166.225.170:35608->104.36.82.13:4204: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168614353 (85h0m31.891177393s) [noise=true , address=104.36.82.49:4011 , piece_id=NL27FJLCGGPHGV2G57FTQXRE6NW3DIRKSTEJKCPT7W33GOWLQALQ, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1DV1WUL3dGBjH23a6xtetmqmLE9fER1FQf53gmYFxZiP25ra8N, address:104.36.82.49:4011): protocol: write tcp 188.166.225.170:51142->104.36.82.49:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168623210 (85h0m31.89118625s) [noise=true , address=104.36.82.55:4212 , piece_id=ECF7CNXJXNVHQM4V6GOTIHNUBH6GW3MNBT5B54Q4EALKHYH7TAJQ, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1EPo99poctfFTjokBfrawXtUeyraudrXQhXFeyQGe72mjD789y, address:104.36.82.55:4212): protocol: write tcp 188.166.225.170:60196->104.36.82.55:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168633139 (85h0m31.891196179s) [noise=true , address=104.36.82.20:4457 , piece_id=SSUFWEWEJTWDCY2RGFOF3JUSKQFSLGKOQCDI4GDSP2VNCOFERCTA, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:16sDrSF9mwW9cQ2h8N5TfUKrnwPAwtEfed6AwCEAT95r5iY9vQ, address:104.36.82.20:4457): protocol: write tcp 188.166.225.170:60762->104.36.82.20:4457: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168640363 (85h0m31.891203393s) [noise=true , address=104.36.82.15:4277 , piece_id=6LWJIDNO5URKOPXM3JLQA4EXLOK67OZJTFLTYMXTMVBZZYGYQZDQ, seg_pos={0 0}, upload=81562]: Done uploading piece. err: ecclient: upload failed (node:1i493KQZuQYQwXzct9HamWDCq92CSGvLPHjZ6fxxVVZX1f7ofY, address:104.36.82.15:4277): protocol: write tcp 188.166.225.170:60122->104.36.82.15:4277: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:36:22.168654239 (85h0m31.891217279s) [seg_pos={0 0}, upload=81562]: Done waiting for segment. successful: 1 optimal: 14 errs: <nil> Jul 28 00:36:22.401018780 (85h0m32.12358182s) [upload=81562]: Done uploading segments... Jul 28 00:36:22.401044589 (85h0m32.123607619s) [upload=81562]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 1) Duration: 1m0.23474114s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................X.X..........X.....X.X..........] duration: 5.894793817s FAIL (2025-07-28 00:36:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.89482165s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............X............X.......................X.........] duration: 1.850946732s FAIL (2025-07-28 00:36:02 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.850975646s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..............................X.......X.XX....X....X.......X] duration: 30.255506767s FAIL (2025-07-28 00:35:58 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.255528738s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................X.X..........X.....X.X.........] duration: 5.767161099s FAIL (2025-07-28 00:35:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.767188652s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 27 out of 60 is failed (threshold=3) [..............................X...XXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.23433276s FAIL (2025-07-28 00:35:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 2) ======LOGS====== , piece_id=RPHSPO2JQMKG6EEL2G4FSCMEESCMQGHJWNMQH6PW5XYELR6PABMQ, seg_pos={0 0}, upload=81541]: Uploading piece... Jul 28 00:34:11.186646093 (84h58m20.909209133s) [noise=true , address=64.49.13.154:4932 , piece_id=ILL43ANVYOBRISC3QMBR6JTCF67DMQG6HC2CNJFJMIPCBTB7KLAQ, seg_pos={0 0}, upload=81541]: Uploading piece... Jul 28 00:34:11.187015289 (84h58m20.909578339s) [noise=true , address=104.36.82.6:4894 , piece_id=AUZEUHAM2XVKRRK5AGS35JKSHVXCPYBYPVB53VZ2UFKPPRHMQHIA, seg_pos={0 0}, upload=81541]: Uploading piece... Jul 28 00:34:11.187865666 (84h58m20.910428706s) [noise=true , address=104.36.82.15:4263 , piece_id=XWWC3B4EX5CE7Q2S5QCJAQJRGQNT4KICA56QMMYJHHPNSQVDQZ7Q, seg_pos={0 0}, upload=81541]: Uploading piece... Jul 28 00:34:11.188578550 (84h58m20.91114159s) [noise=true , address=104.36.82.14:4242 , piece_id=6XHODN4FXR4L7KDIVRI7XZ43JSQBD6E26HWR5DXS6XBTNWPNKUFA, seg_pos={0 0}, upload=81541]: Uploading piece... Jul 28 00:34:11.193602393 (84h58m20.916165433s) [noise=true , address=104.36.82.3:4783 , piece_id=C3E35WFHLV6QATOFQNFEEH3CCITV35TOUOKTQ5BWSVQLKWXKHQDA, seg_pos={0 0}, upload=81541]: Uploading piece... Jul 28 00:34:11.194456497 (84h58m20.917019537s) [noise=true , address=104.36.82.51:4059 , piece_id=WAAFHIH5XM6WT7N3WLZ3YLTOMPILZADKLJQYZEWLHX2ENRJ5VCMA, seg_pos={0 0}, upload=81541]: Uploading piece... Jul 28 00:34:12.331374429 (84h58m22.053937469s) [upload=81541]: Next returned nil segment Jul 28 00:34:12.331391842 (84h58m22.053954883s) [upload=81541]: Waiting for error group managing segments... Jul 28 00:34:12.640722704 (84h58m22.363285744s) [noise=true , address=64.49.13.154:4932 , piece_id=ILL43ANVYOBRISC3QMBR6JTCF67DMQG6HC2CNJFJMIPCBTB7KLAQ, seg_pos={0 0}, upload=81541]: Done uploading piece. err: <nil> Jul 28 00:34:13.747173143 (84h58m23.469736183s) [noise=true , address=77.90.31.16:4748 , piece_id=5PMV6PPODQQENZMCG7OXUKI7Q36XJI7JDX6D5B2ULEQWG5XJJE6A, seg_pos={0 0}, upload=81541]: Done uploading piece. err: <nil> Jul 28 00:35:10.081128191 (84h59m19.803691241s) [noise=true , address=104.36.82.35:4615 , piece_id=PDBNLHVRWOGD6VQRYUXRARQFJKXBUANFL6YAUAJHXLFRPBNO23LA, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:12NJ5w8pAWCudDA9Xs4ES7b2evnbscesd682wJDh2gMMZ3Jf2zc, address:104.36.82.35:4615): protocol: write tcp 188.166.225.170:50616->104.36.82.35:4615: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081179818 (84h59m19.803742859s) [noise=true , address=104.36.82.13:4193 , piece_id=RPHSPO2JQMKG6EEL2G4FSCMEESCMQGHJWNMQH6PW5XYELR6PABMQ, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:15XNLgptmXprcfkVMJsuzY2WDyHiZDpLetiBpmjXhb4c9q7yMd, address:104.36.82.13:4193): protocol: write tcp 188.166.225.170:60864->104.36.82.13:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081214205 (84h59m19.803777235s) [noise=true , address=104.36.82.6:4894 , piece_id=AUZEUHAM2XVKRRK5AGS35JKSHVXCPYBYPVB53VZ2UFKPPRHMQHIA, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:12fZsj77XZ8CnYD6iwzEVKPn6yLyxWMtibzsRXcuGKwNxEXB6Hq, address:104.36.82.6:4894): protocol: write tcp 188.166.225.170:38976->104.36.82.6:4894: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081249612 (84h59m19.803812653s) [noise=true , address=104.36.82.53:4154 , piece_id=LMMA7JHS2NGKKX3OB3MNWYCM3UPNGPQLXS347PCXYDLLPBNZWOOQ, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:1vbGunXCRMkmfvBqHPmbuW7fBLKck4Vh9MGsX1q7P6VfX6DKqB, address:104.36.82.53:4154): protocol: write tcp 188.166.225.170:52900->104.36.82.53:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081281223 (84h59m19.803844253s) [noise=true , address=104.36.82.12:4154 , piece_id=GQYNCE7AITKLZ7DOHQNZ7LYKEXAJVI2FMUT6IBXCJPIL3UGFM36A, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:12S7Xsfnti3QuD9apBq1sgSjmahnAYA6LYbXqyckbLY4UU84ZQF, address:104.36.82.12:4154): protocol: write tcp 188.166.225.170:47294->104.36.82.12:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081313064 (84h59m19.803876094s) [noise=true , address=104.36.82.55:4193 , piece_id=HYOCXSRM47ZMO7QUU35JTPU3GVLGSF6TWWEB3ZEUUD3YHFFO5VVA, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, address:104.36.82.55:4193): protocol: rpc: dial tcp 104.36.82.55:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081347079 (84h59m19.803910109s) [noise=true , address=104.36.82.14:4242 , piece_id=6XHODN4FXR4L7KDIVRI7XZ43JSQBD6E26HWR5DXS6XBTNWPNKUFA, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:104.36.82.14:4242): protocol: write tcp 188.166.225.170:59762->104.36.82.14:4242: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081382457 (84h59m19.803945487s) [noise=true , address=104.36.82.51:4059 , piece_id=WAAFHIH5XM6WT7N3WLZ3YLTOMPILZADKLJQYZEWLHX2ENRJ5VCMA, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:12fQsGGr5K6TVMU7XC2v1mQBFwHznj5ugYvYLmmAUnP81AyHAjY, address:104.36.82.51:4059): protocol: write tcp 188.166.225.170:33624->104.36.82.51:4059: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081416692 (84h59m19.803979732s) [noise=true , address=104.36.82.54:4191 , piece_id=FJXF54KAMXXFSJPM5DGH56RTPCSLTGMFEBBJRTK3FLK4UIF4VKDQ, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:12d7ChCRcN4pJjJ51aSAPitg3zTRxr5eVbRXVZcvbi6t1gifWmw, address:104.36.82.54:4191): protocol: rpc: dial tcp 104.36.82.54:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081457481 (84h59m19.804020521s) [noise=true , address=104.36.82.3:4783 , piece_id=C3E35WFHLV6QATOFQNFEEH3CCITV35TOUOKTQ5BWSVQLKWXKHQDA, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:12ito72D7vikRdjK7BPt4LnhAK84bzi8AbZZPNWrTqmbjwQT1N, address:104.36.82.3:4783): protocol: write tcp 188.166.225.170:42850->104.36.82.3:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081493920 (84h59m19.80405696s) [noise=true , address=104.36.82.52:4090 , piece_id=73SMTCFCCI76TX3TG2SYNSDPTBS5NOCKEG5HAVKKB7DKNSY464ZA, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:12UUnoD9p5FC7rq9fMSzu1phkFpsnPsu4UYxC59MhpMjJ6wUC6F, address:104.36.82.52:4090): protocol: write tcp 188.166.225.170:42426->104.36.82.52:4090: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081533115 (84h59m19.804096155s) [noise=true , address=104.36.82.12:4169 , piece_id=443NIAZIZBAABGE64P7H5RC3J4VN3EBYEVOLDV6UIZFDR3NOSJIA, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: write tcp 188.166.225.170:35794->104.36.82.12:4169: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081550899 (84h59m19.804113939s) [noise=true , address=104.36.82.13:4220 , piece_id=WPBF76CZLFTULBAUXCRUKY7NUB65GXFWIDD3YF6ADASPSD723E6A, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:1AjQmw3H58hTwr2389cmMcwy1gBavBrZB8AWB1gqfoFRUkCpH6, address:104.36.82.13:4220): protocol: write tcp 188.166.225.170:33048->104.36.82.13:4220: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081578412 (84h59m19.804141442s) [noise=true , address=104.36.82.42:4818 , piece_id=T4NRYW7NX2KKQTCXRWAGUJOY354GLZEMKHHYEABPSJZ2TXNP47IQ, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:122m1dk7cCwZfUKeq4BChF1UjXSt7swvt1L3HdiSybavXhLEcsb, address:104.36.82.42:4818): protocol: write tcp 188.166.225.170:55632->104.36.82.42:4818: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081616073 (84h59m19.804179113s) [noise=true , address=104.36.82.56:4233 , piece_id=3VTZ3QJO5AYNKPGJX34ZN6QGJFXGMDO4J7MPJ4K3ISOK3PS3ARHQ, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:1DmQeoQWTnFhQeJFseJoNYzFyo7vZjYKwDtQVJ6pomksWExsTb, address:104.36.82.56:4233): protocol: write tcp 188.166.225.170:46066->104.36.82.56:4233: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081644908 (84h59m19.804207948s) [noise=true , address=104.36.82.15:4263 , piece_id=XWWC3B4EX5CE7Q2S5QCJAQJRGQNT4KICA56QMMYJHHPNSQVDQZ7Q, seg_pos={0 0}, upload=81541]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:45518->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:35:10.081692550 (84h59m19.80425559s) [seg_pos={0 0}, upload=81541]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil> Jul 28 00:35:10.313368144 (84h59m20.035931184s) [upload=81541]: Done uploading segments... Jul 28 00:35:10.313395426 (84h59m20.035958466s) [upload=81541]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2) Duration: 1m0.234345685s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X............X.......................X........] duration: 1.962413981s FAIL (2025-07-28 00:35:04 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.962447054s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................X.X..........X.....X.X........] duration: 5.841326255s FAIL (2025-07-28 00:34:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.841350662s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................X.......X.XX....X....X.......] duration: 5.717905685s FAIL (2025-07-28 00:34:32 UTC) orbiter warning: digest mismatch: after 6b93ca9819ba9c70b065b54057b3f984abfbe2ed3f41c7f9ca09ec5e2ec8ba56 != before bfb4991c3c789f3b76f90db2ddfb43319ad13805292a8a70202450851dea6594 Duration: 5.717930061s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............................X.X..........X.....X.X.......] duration: 5.846990585s FAIL (2025-07-28 00:34:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.847018258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............X............X.......................X.......] duration: 1.887216839s FAIL (2025-07-28 00:34:30 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.88724369s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 26 out of 60 is failed (threshold=3) [...............................X...XXXXXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.236287612s FAIL (2025-07-28 00:34:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 3) ======LOGS====== 9s) [noise=true , address=104.36.82.41:4792 , piece_id=SYECFTFRXUXTV3AH6S5ZCCJDJ4FK55QSM3AZGBJI2KYJQJOCRG6A, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.698765679 (84h57m20.421328719s) [noise=true , address=104.36.82.54:4178 , piece_id=6K67LI7ORQPN55ZAFMYCLERDPELOBWD23J7MLY75BV5RI5OECLVQ, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.698981862 (84h57m20.421544892s) [noise=true , address=104.36.82.10:4094 , piece_id=RRAIL44JAI3S7YKHWDKWSAXASHUKUUIE3LCTQNJ4VBAFEPR2UMNQ, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.699198977 (84h57m20.421762017s) [noise=true , address=79.127.205.235:4680 , piece_id=IF37CXAAWWSYVS35WBU673TAOYAYOHCYMXSWKP54V2POL6PLLGRQ, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.699336190 (84h57m20.42189923s) [noise=true , address=104.36.82.12:4173 , piece_id=QA3LG5NAHCK6WN5CB3RNJHTUU2LFALXIIGLLV3NPN4NOYDRBGJAQ, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.699590717 (84h57m20.422153767s) [noise=true , address=104.36.82.49:4996 , piece_id=OXYDIPLEPI75BALZQP4WXT4Z624EZGHN5PXGGRAHQDWZRP7HHQPQ, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.699803113 (84h57m20.422366153s) [noise=true , address=104.36.82.56:4263 , piece_id=HCBLSZYV2FB3G2DMO6J2CAAIXASU753PCB5CDWT5O4J7SF2JNJKQ, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.700089311 (84h57m20.422652351s) [noise=true , address=104.36.82.12:4164 , piece_id=GR5PC75RIVY2GFKLH2BYRT6LJUZQ2A3UJJWZ5H7VI5MD62PYJDKQ, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.700251562 (84h57m20.422814602s) [noise=true , address=104.36.82.5:4865 , piece_id=KPDVEP3BOJBSESMSVFXLDRAQCTBJIQIJVAXQF7J67G2KJYP46MPA, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:10.700531728 (84h57m20.423094768s) [noise=true , address=104.36.82.11:4130 , piece_id=QDL2JDDN66JIH3WJK2SSH4TNGLEAULE4QCWAO3FUQP5IW3B7SGPA, seg_pos={0 0}, upload=81524]: Uploading piece... Jul 28 00:33:11.826597687 (84h57m21.549160727s) [upload=81524]: Next returned nil segment Jul 28 00:33:11.826616423 (84h57m21.549179463s) [upload=81524]: Waiting for error group managing segments... Jul 28 00:33:12.248980612 (84h57m21.971543652s) [noise=true , address=79.127.205.235:4680 , piece_id=IF37CXAAWWSYVS35WBU673TAOYAYOHCYMXSWKP54V2POL6PLLGRQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: <nil> Jul 28 00:33:16.395924860 (84h57m26.1184879s) [noise=true , address=64.49.13.140:4508 , piece_id=LL5WBCBIBQBYD2YJU6YLGM7KNMDAFPZWCJZ5RLHZ6JS46762VDYQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: <nil> Jul 28 00:33:30.687648510 (84h57m40.41021155s) [noise=true , address=104.36.82.5:4865 , piece_id=KPDVEP3BOJBSESMSVFXLDRAQCTBJIQIJVAXQF7J67G2KJYP46MPA, seg_pos={0 0}, upload=81524]: Done uploading piece. err: <nil> Jul 28 00:34:09.609715024 (84h58m19.332278084s) [noise=true , address=104.36.82.56:4263 , piece_id=HCBLSZYV2FB3G2DMO6J2CAAIXASU753PCB5CDWT5O4J7SF2JNJKQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:12tLW4r8AQKLCbSWZoNUVi5v2YtF8NthUmX81TLQn8G8TotuByG, address:104.36.82.56:4263): protocol: rpc: dial tcp 104.36.82.56:4263: i/o timeout; context deadline exceeded Jul 28 00:34:09.610471942 (84h58m19.333035022s) [noise=true , address=104.36.82.11:4130 , piece_id=QDL2JDDN66JIH3WJK2SSH4TNGLEAULE4QCWAO3FUQP5IW3B7SGPA, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:12n69riZBquh3Qi436LVfVgNDJYuDxPzGi7d2BZDLurUJYVMCqs, address:104.36.82.11:4130): protocol: rpc: dial tcp 104.36.82.11:4130: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.610611318 (84h58m19.333174368s) [noise=true , address=104.36.82.52:4106 , piece_id=CEHG3GGYKBL6ZSHOJZKPGOUSPVGSR5765L7FZIMOC4TOSXCC2OZQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:12E2hMuAUbJyh5F4yey1EtRLYhzr29wxs47qY3zGxmGvFNUctk2, address:104.36.82.52:4106): protocol: rpc: dial tcp 104.36.82.52:4106: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.610671483 (84h58m19.333234563s) [noise=true , address=104.36.82.14:4247 , piece_id=3XEEW3WS2PRADEGMF7ZKOIBRK7FD2UN6WWUYYEBQ5FEKUCQEUBNA, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:1262xxX6aUPTYgKdvRxAyc77PmPBWJtq5cXRJFoELxHbfn18DLE, address:104.36.82.14:4247): protocol: rpc: dial tcp 104.36.82.14:4247: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.610722090 (84h58m19.33328514s) [noise=true , address=104.36.82.41:4792 , piece_id=SYECFTFRXUXTV3AH6S5ZCCJDJ4FK55QSM3AZGBJI2KYJQJOCRG6A, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:12pynuFncf2qop1yNZvkk9a1NsWPrvpYvwUnwXVcvKUQgpHMC2k, address:104.36.82.41:4792): protocol: rpc: dial tcp 104.36.82.41:4792: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.610781313 (84h58m19.333344353s) [noise=true , address=104.36.82.10:4094 , piece_id=RRAIL44JAI3S7YKHWDKWSAXASHUKUUIE3LCTQNJ4VBAFEPR2UMNQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:124zBCfZBTW69BSaGRzUyjWUBjSJhaLMZXQYnqMPGSKL4EX6HVV, address:104.36.82.10:4094): protocol: rpc: dial tcp 104.36.82.10:4094: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.610840718 (84h58m19.333403758s) [noise=true , address=104.36.82.16:4326 , piece_id=GBPPEWEQD4D7XDYHUDHPEFQJWTAWE4MAXMTKHARSUQCBCDFMBMCQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:12pU8MZZKS88CPFdZoaQ4msNfkCbCewTmS2wKEdN7pG5aCWoAr6, address:104.36.82.16:4326): protocol: write tcp 188.166.225.170:53392->104.36.82.16:4326: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.610884351 (84h58m19.333447391s) [noise=true , address=104.36.82.20:4441 , piece_id=KXJP6GVDT7FA5BA4ZNK4D24I7BOJY34SMYP6WRECL5JWKCMOFLJQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:12LjGecRNJE1kq67iMtxZFhWeaUZpUogsGxbz4VXwnHdQtTnbBr, address:104.36.82.20:4441): protocol: rpc: dial tcp 104.36.82.20:4441: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.611012045 (84h58m19.333575085s) [noise=true , address=104.36.82.53:4135 , piece_id=GPY2PZVGUSYDVBBXIQB2PKLKW63OT3BHC5QJ72Y6OSRMBRGJYK6Q, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:128j2QoFq4UDWCXjV8uChmN9KB1sQMnhdF6hGgNQygdqHvw9tPj, address:104.36.82.53:4135): protocol: write tcp 188.166.225.170:50118->104.36.82.53:4135: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.611055538 (84h58m19.333618588s) [noise=true , address=104.36.82.12:4173 , piece_id=QA3LG5NAHCK6WN5CB3RNJHTUU2LFALXIIGLLV3NPN4NOYDRBGJAQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:12HbGUhwH2eNqY75sLQCDgkvJ1QSrJ2n4Qg8A5Z73dFXxRpVwSC, address:104.36.82.12:4173): protocol: rpc: dial tcp 104.36.82.12:4173: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.611085596 (84h58m19.333648647s) [noise=true , address=104.36.82.12:4164 , piece_id=GR5PC75RIVY2GFKLH2BYRT6LJUZQ2A3UJJWZ5H7VI5MD62PYJDKQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:12iQgyDiz9vu1TbCRuATGmnrfbgBaia72b77ZjQJ6gpnEFbQzTJ, address:104.36.82.12:4164): protocol: rpc: dial tcp 104.36.82.12:4164: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.611097218 (84h58m19.333660268s) [noise=true , address=104.36.82.49:4996 , piece_id=OXYDIPLEPI75BALZQP4WXT4Z624EZGHN5PXGGRAHQDWZRP7HHQPQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:1qomJ2ceswE6SKfoPAxWpiCkcW88EpfFbPnJXs7ETb5CZvSpqS, address:104.36.82.49:4996): protocol: rpc: dial tcp 104.36.82.49:4996: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.611133578 (84h58m19.333696618s) [noise=true , address=104.36.82.7:4944 , piece_id=DQCQME2PIXJ4UDP4N4V6UBE2VEJDV7ZWSCW2QC45JMDM2UZIJJQQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:15mgZMkU28J1JwrRd4XbvZFyjp8G7CTBkJCxbzUYqKeSeqCZ9F, address:104.36.82.7:4944): protocol: rpc: dial tcp 104.36.82.7:4944: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.611141002 (84h58m19.333704043s) [noise=true , address=104.36.82.54:4178 , piece_id=6K67LI7ORQPN55ZAFMYCLERDPELOBWD23J7MLY75BV5RI5OECLVQ, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:1jyRCtxSCu2bkU41Kgb1aeGppQLgX5iEo322Zoi1cWFkmGeYgr, address:104.36.82.54:4178): protocol: write tcp 188.166.225.170:40244->104.36.82.54:4178: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.611170579 (84h58m19.333733629s) [noise=true , address=104.36.82.55:4193 , piece_id=MV2WZRSVVX5AWC26YJ5IWSZLZZC6J7DYZLOMYYDBG7SX572HLKCA, seg_pos={0 0}, upload=81524]: Done uploading piece. err: ecclient: upload failed (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, address:104.36.82.55:4193): protocol: write tcp 188.166.225.170:46240->104.36.82.55:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:34:09.611186980 (84h58m19.33375003s) [seg_pos={0 0}, upload=81524]: Done waiting for segment. successful: 3 optimal: 14 errs: <nil> Jul 28 00:34:09.844596272 (84h58m19.567159322s) [upload=81524]: Done uploading segments... Jul 28 00:34:09.844633002 (84h58m19.567196042s) [upload=81524]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 3) Duration: 1m0.236300356s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [................X............X.......................X......] duration: 1.823046592s FAIL (2025-07-28 00:34:01 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.823074476s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................X.......X.XX....X....X......] duration: 5.809342664s FAIL (2025-07-28 00:33:54 UTC) orbiter warning: digest mismatch: after 6b93ca9819ba9c70b065b54057b3f984abfbe2ed3f41c7f9ca09ec5e2ec8ba56 != before bfb4991c3c789f3b76f90db2ddfb43319ad13805292a8a70202450851dea6594 Duration: 5.809368495s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................................X.X..........X.....X.X......] duration: 6.025388043s FAIL (2025-07-28 00:33:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.025417068s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................X.......X.XX....X....X.....] duration: 5.939563164s FAIL (2025-07-28 00:33:11 UTC) orbiter warning: digest mismatch: after 6b93ca9819ba9c70b065b54057b3f984abfbe2ed3f41c7f9ca09ec5e2ec8ba56 != before bfb4991c3c789f3b76f90db2ddfb43319ad13805292a8a70202450851dea6594 Duration: 5.939586377s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 25 out of 60 is failed (threshold=3) [................................X...XXXXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.232325616s FAIL (2025-07-28 00:33:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 2) ======LOGS====== ]: Uploading piece... Jul 28 00:32:10.247040697 (84h56m19.969603727s) [noise=true , address=104.36.82.11:4130 , piece_id=VYUO44ZM5IOUADNGVGOORTCDTYTDC7LVZNBTHYWLJCCVAR3TPUNQ, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:10.247281317 (84h56m19.969844357s) [noise=true , address=104.36.82.50:4030 , piece_id=JZWXUSRMZ6HHEQADD5J24QD2KNS4RAE6UFZQ26RRK6VD6DCEF6BQ, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:10.247305313 (84h56m19.969868353s) [noise=true , address=104.36.82.42:4817 , piece_id=ZLQ2EQ3356DTP2BTE7OFN25EXPW3VKPDM6R4SUVQLTWG2IKP3VFQ, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:10.247511738 (84h56m19.970074778s) [noise=true , address=104.36.82.15:4263 , piece_id=ZCVEL5HGPTXWWMPJBNLBFNZWE3ZBHU6PJ3XUS2UNLZCYEY2HIAAA, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:10.247536836 (84h56m19.970099876s) [noise=true , address=104.36.82.36:4632 , piece_id=4O3MSSD6D2IUZMFTJKUNDI6W3MZ7G6UQEZD7YPY4GHXBLA6SSNSA, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:10.247825278 (84h56m19.970388318s) [noise=true , address=64.49.13.138:4482 , piece_id=GSLBSUVOYRBBDT2WKXTCFNGKU6LOOOBYHYF4PJOLK4JFCQHSTX3A, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:10.248058524 (84h56m19.970621564s) [noise=true , address=104.36.82.14:4245 , piece_id=SFMTOCGS5RN56AOIWSZQA5KCBYYIZAMSYPFB6AFLTQETF4GVUXOA, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:10.248285448 (84h56m19.970848488s) [noise=true , address=104.36.82.47:4517 , piece_id=7HMOHZQLHTVAVR3DZNQ6L45A7732BAXA5XPAXLK2HKM7XP473TUQ, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:10.248562929 (84h56m19.971125969s) [noise=true , address=79.127.201.215:4347 , piece_id=N5PAJS3TL6VCVQM5HGQU7OVOT4GSMKOGAYMB6Q3O4FVWLTRMFXCQ, seg_pos={0 0}, upload=81507]: Uploading piece... Jul 28 00:32:11.392616244 (84h56m21.115179284s) [upload=81507]: Next returned nil segment Jul 28 00:32:11.392633728 (84h56m21.115196768s) [upload=81507]: Waiting for error group managing segments... Jul 28 00:32:12.055979967 (84h56m21.778543007s) [noise=true , address=64.49.13.138:4482 , piece_id=GSLBSUVOYRBBDT2WKXTCFNGKU6LOOOBYHYF4PJOLK4JFCQHSTX3A, seg_pos={0 0}, upload=81507]: Done uploading piece. err: <nil> Jul 28 00:32:12.253720744 (84h56m21.976283784s) [noise=true , address=79.127.201.215:4347 , piece_id=N5PAJS3TL6VCVQM5HGQU7OVOT4GSMKOGAYMB6Q3O4FVWLTRMFXCQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: <nil> Jul 28 00:33:09.147055765 (84h57m18.869618795s) [noise=true , address=104.36.82.50:4030 , piece_id=JZWXUSRMZ6HHEQADD5J24QD2KNS4RAE6UFZQ26RRK6VD6DCEF6BQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1fRTqHVZiYV7NBAp7PLEPVXGEDZ7VXxQJuRhkXkAkNpT7ra27e, address:104.36.82.50:4030): protocol: rpc: dial tcp 104.36.82.50:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147127042 (84h57m18.869690082s) [noise=true , address=104.36.82.9:4009 , piece_id=C5AKHDCSQTO2MFAZKFWU6ONY7QARLLSDBVWI6664FUZ7HMLN6XRQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:12n33Xie5Hk1vaKc279HvGnL2QBEYsNzieL2grpnANJpy7Qu2wb, address:104.36.82.9:4009): protocol: rpc: dial tcp 104.36.82.9:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147160305 (84h57m18.869723345s) [noise=true , address=104.36.82.55:4193 , piece_id=3SE7KZRBBMRPDHKE3AXRXINAF3HA5GHXXOJQC5GC3FPPQ77SXZQQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, address:104.36.82.55:4193): protocol: rpc: dial tcp 104.36.82.55:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147188800 (84h57m18.86975183s) [noise=true , address=104.36.82.47:4517 , piece_id=7HMOHZQLHTVAVR3DZNQ6L45A7732BAXA5XPAXLK2HKM7XP473TUQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, address:104.36.82.47:4517): protocol: rpc: dial tcp 104.36.82.47:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147219478 (84h57m18.869782518s) [noise=true , address=104.36.82.43:4868 , piece_id=RKAR33A2AALB3KK6OFGC3EJ6I7B6NCSA3AAUXS3FHADSS6ZLXFVA, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:124LJtYshPwu1uZJwVyjrjUMk3htyVnp9ntnZWSUb7X3bNkwU6t, address:104.36.82.43:4868): protocol: rpc: dial tcp 104.36.82.43:4868: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147248574 (84h57m18.869811614s) [noise=true , address=104.36.82.45:4936 , piece_id=2NH3CNWI5GZROOG5LQ5K4R5OUFWXB4CC6RDW3OQBL3K4NSRQVHTA, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1DicJh1hd5NUoj9uxoqzeaCmjGHwfk2VUaA9zQFmv72wBdBoW8, address:104.36.82.45:4936): protocol: rpc: dial tcp 104.36.82.45:4936: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147275666 (84h57m18.869838706s) [noise=true , address=104.36.82.53:4147 , piece_id=HF6QO6SDK2ZREDIUY5R76PVPG3DLFY3JWFAGLIKVJMN4HUNMFNGQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:12pg6fGWqRjS8Gf3zQgw8jbuytaCyMpJ2CAz4oRVHYdLAhttgnH, address:104.36.82.53:4147): protocol: rpc: dial tcp 104.36.82.53:4147: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147301094 (84h57m18.869864134s) [noise=true , address=104.36.82.20:4454 , piece_id=5AKBK5Z2HCDV727LZZOT2AMWO2XFXSNSYKXU6EMENBZFLLJWTAIA, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1SafkJPqKHLwMaqyZdLq3yJ1xdjqDtWHFrUwynmegUVwEC7y5J, address:104.36.82.20:4454): protocol: rpc: dial tcp 104.36.82.20:4454: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147328767 (84h57m18.869891807s) [noise=true , address=104.36.82.6:4885 , piece_id=MRXBZKBIG34JYNKUFNYPH2YRRQ77DHZ2OKQUN7RCOYEDR6GCRM3Q, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:12rLd1AWyRtEVFmK4iJCebp5VAJEjnmhhgtUQcV1FB8vMvRnUhm, address:104.36.82.6:4885): protocol: rpc: dial tcp 104.36.82.6:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147360017 (84h57m18.869923057s) [noise=true , address=104.36.82.42:4817 , piece_id=ZLQ2EQ3356DTP2BTE7OFN25EXPW3VKPDM6R4SUVQLTWG2IKP3VFQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1w1W2uV2kFMfxwSwB5o1rZbd4VGVR2XA7sk3NLWRQcZoBnheE7, address:104.36.82.42:4817): protocol: rpc: dial tcp 104.36.82.42:4817: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147388000 (84h57m18.869951041s) [noise=true , address=104.36.82.15:4263 , piece_id=ZCVEL5HGPTXWWMPJBNLBFNZWE3ZBHU6PJ3XUS2UNLZCYEY2HIAAA, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: rpc: dial tcp 104.36.82.15:4263: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147415945 (84h57m18.869978985s) [noise=true , address=104.36.82.36:4632 , piece_id=4O3MSSD6D2IUZMFTJKUNDI6W3MZ7G6UQEZD7YPY4GHXBLA6SSNSA, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1B7uLDPBrYvc1N81dKm8bcYnhYwjA5mRdRfeQpWqKqrJjmQJgC, address:104.36.82.36:4632): protocol: rpc: dial tcp 104.36.82.36:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147443468 (84h57m18.870006508s) [noise=true , address=104.36.82.54:4165 , piece_id=XCPWPVRFMF24SLMII6GKHLDKSY3MOLJWHLTZFYS6XXDUUV5AXIYQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:12wGUMXVEubEtki4qGnjGEsbYFB1iUauZcu3oDRKrCQsJMV3bV2, address:104.36.82.54:4165): protocol: rpc: dial tcp 104.36.82.54:4165: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147468785 (84h57m18.870031825s) [noise=true , address=104.36.82.41:4762 , piece_id=HAFZBR3IO3TS6APFB36QQMHA4GNXANW36R662MQR4STZWDMABHAQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:104.36.82.41:4762): protocol: rpc: dial tcp 104.36.82.41:4762: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147494705 (84h57m18.870057745s) [noise=true , address=104.36.82.11:4130 , piece_id=VYUO44ZM5IOUADNGVGOORTCDTYTDC7LVZNBTHYWLJCCVAR3TPUNQ, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:12n69riZBquh3Qi436LVfVgNDJYuDxPzGi7d2BZDLurUJYVMCqs, address:104.36.82.11:4130): protocol: rpc: dial tcp 104.36.82.11:4130: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147526757 (84h57m18.870089797s) [noise=true , address=104.36.82.14:4245 , piece_id=SFMTOCGS5RN56AOIWSZQA5KCBYYIZAMSYPFB6AFLTQETF4GVUXOA, seg_pos={0 0}, upload=81507]: Done uploading piece. err: ecclient: upload failed (node:1Zu8teNPTXn638grHKwtJqz4UEY7oFYLx1DHvuVP6RZXcatVqX, address:104.36.82.14:4245): protocol: write tcp 188.166.225.170:48656->104.36.82.14:4245: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:33:09.147540332 (84h57m18.870103372s) [seg_pos={0 0}, upload=81507]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil> Jul 28 00:33:09.376631782 (84h57m19.099194882s) [upload=81507]: Done uploading segments... Jul 28 00:33:09.376685816 (84h57m19.099248856s) [upload=81507]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2) Duration: 1m0.232349221s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................X.X..........X.....X.X.....] duration: 5.76470518s FAIL (2025-07-28 00:32:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.76473707s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................X............X.......................X.....] duration: 1.896496482s FAIL (2025-07-28 00:32:46 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.896522303s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................X............X.......................X....] duration: 1.854090328s FAIL (2025-07-28 00:32:26 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.854123491s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................................X.......X.XX....X....X....] duration: 6.029700435s FAIL (2025-07-28 00:32:25 UTC) orbiter warning: digest mismatch: after 6b93ca9819ba9c70b065b54057b3f984abfbe2ed3f41c7f9ca09ec5e2ec8ba56 != before bfb4991c3c789f3b76f90db2ddfb43319ad13805292a8a70202450851dea6594 Duration: 6.029727807s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................X............X.......................X...] duration: 1.952155201s FAIL (2025-07-28 00:32:09 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.95217552s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 24 out of 60 is failed (threshold=3) [.................................X...XXXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.232779669s FAIL (2025-07-28 00:32:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 1) ======LOGS====== 55m19.503647909s) [noise=true , address=104.36.82.11:4116 , piece_id=BUBZZJU3M7VHXKVLSU65RP7LF4CR7U2ZNUV7I7NJ2DBIOAJX6I7Q, seg_pos={0 0}, upload=81489]: Uploading piece... Jul 28 00:31:09.781496807 (84h55m19.504059857s) [noise=true , address=104.36.82.5:4850 , piece_id=GRA6TIREH63NIANSQABVYAWBLKNSIYO3I3NOQHO2OG6ZHMPEY5HA, seg_pos={0 0}, upload=81489]: Uploading piece... Jul 28 00:31:09.782085443 (84h55m19.504648483s) [noise=true , address=104.36.82.49:4990 , piece_id=EOYPY2SZKA6UZ66RNOTQSCLQQFUSPBPSV7ICWYHWRMIDMQ7SHNDQ, seg_pos={0 0}, upload=81489]: Uploading piece... Jul 28 00:31:09.782091945 (84h55m19.504654995s) [noise=true , address=104.36.82.12:4165 , piece_id=XCEP2UUHZ5RRH2GVL22VPMSFLKRJEGHT2I4CP5OPYS3V52C4NDCQ, seg_pos={0 0}, upload=81489]: Uploading piece... Jul 28 00:31:09.782324030 (84h55m19.50488707s) [noise=true , address=79.127.219.38:4001 , piece_id=RGTDTRMJMUTIFKVK2P47XPNVWKF2YYU22TV3B5LMRMRXVURMZ2KA, seg_pos={0 0}, upload=81489]: Uploading piece... Jul 28 00:31:09.782572886 (84h55m19.505135936s) [noise=true , address=104.36.82.21:4484 , piece_id=5NXUK4G5OLOOO2NW5XYYN7SES5C7HQ5CEFVBG6YS6GAG6JGPKURA, seg_pos={0 0}, upload=81489]: Uploading piece... Jul 28 00:31:09.782790152 (84h55m19.505353192s) [noise=true , address=104.36.82.3:4774 , piece_id=6VDWT6HHR7NKG5SVO5FSJGB7L2WY7NXBGBNRTDKUGHYZSKSWMIZA, seg_pos={0 0}, upload=81489]: Uploading piece... Jul 28 00:31:09.783024480 (84h55m19.50558752s) [noise=true , address=104.36.82.16:4307 , piece_id=Z3QSDZTU2GWIEEMBX5V3PDS5DXJ6KSDJSSBQZYPXMJ6SUVNY4GQQ, seg_pos={0 0}, upload=81489]: Uploading piece... Jul 28 00:31:11.331208188 (84h55m21.053771228s) [upload=81489]: Next returned nil segment Jul 28 00:31:11.331229709 (84h55m21.053792749s) [upload=81489]: Waiting for error group managing segments... Jul 28 00:31:11.774734496 (84h55m21.497297536s) [noise=true , address=79.127.219.38:4001 , piece_id=RGTDTRMJMUTIFKVK2P47XPNVWKF2YYU22TV3B5LMRMRXVURMZ2KA, seg_pos={0 0}, upload=81489]: Done uploading piece. err: <nil> Jul 28 00:32:08.681757995 (84h56m18.404321035s) [noise=true , address=104.36.82.16:4307 , piece_id=Z3QSDZTU2GWIEEMBX5V3PDS5DXJ6KSDJSSBQZYPXMJ6SUVNY4GQQ, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:12QGfzeTEErAaE5YRTLi1w7SZoJbo2xVKb1Sykj1pW5fbCZYSCn, address:104.36.82.16:4307): protocol: rpc: dial tcp 104.36.82.16:4307: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.681833289 (84h56m18.404396329s) [noise=true , address=104.36.82.51:4050 , piece_id=EDYL3VAHLL4TFHQ6JAQALHLDVZOWELTRHPYW5VV6CKVUEQNJ222Q, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:12Vw9PtVFLQPpm2uGV55tCoK7VfopaBU4EcWNBUyqhNeJjgjG64, address:104.36.82.51:4050): protocol: rpc: dial tcp 104.36.82.51:4050: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.681883946 (84h56m18.404446986s) [noise=true , address=104.36.82.9:4008 , piece_id=NU4H6PC4C355SHYUBDEYAAJ2EKOUXZKTOFPBCFVGTRK7R2O5PFIQ, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:148CqQmrsJSfoGByHYRPoZmvP1VgQQpaMVLb7RgRQvNBuo7air, address:104.36.82.9:4008): protocol: rpc: dial tcp 104.36.82.9:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.681916718 (84h56m18.404479758s) [noise=true , address=104.36.82.47:4540 , piece_id=56Z54VS2WTWVSIETQOYV3OA5WUSRRPEYTHLNZBZFVMJ4KEPRMC5Q, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:104.36.82.47:4540): protocol: rpc: dial tcp 104.36.82.47:4540: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.681948459 (84h56m18.404511499s) [noise=true , address=104.36.82.49:4990 , piece_id=EOYPY2SZKA6UZ66RNOTQSCLQQFUSPBPSV7ICWYHWRMIDMQ7SHNDQ, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:1gDzCKcMaLTisHw68GYkMbcMJF7rbKnUt48etS8Q7KeiXedbU4, address:104.36.82.49:4990): protocol: rpc: dial tcp 104.36.82.49:4990: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682046897 (84h56m18.404609937s) [noise=true , address=104.36.82.12:4165 , piece_id=XCEP2UUHZ5RRH2GVL22VPMSFLKRJEGHT2I4CP5OPYS3V52C4NDCQ, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:13Ksy7YCGV5u3T5Z6WwKtMjFq6Rs4bayzR7mQFyXbSFuJjaH4F, address:104.36.82.12:4165): protocol: rpc: dial tcp 104.36.82.12:4165: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682103395 (84h56m18.404666435s) [noise=true , address=104.36.82.52:4119 , piece_id=YMJMECNMEDBIFRBYLGEHJCCQADXWF5YBV2XRY6XLNNTANBKJQWHQ, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG, address:104.36.82.52:4119): protocol: rpc: dial tcp 104.36.82.52:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682167348 (84h56m18.404730378s) [noise=true , address=104.36.82.41:4786 , piece_id=F32Y2QH7M65BU7LCRRVGHOKWG76GE5O3IGSMSHHENGAJROCYS5GQ, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:125TWU7R4pkmrUxsNLhgTD475aLbjUJ7psA6drJtE7oDqZXYJ4Q, address:104.36.82.41:4786): protocol: rpc: dial tcp 104.36.82.41:4786: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682248964 (84h56m18.404812004s) [noise=true , address=104.36.82.54:4190 , piece_id=MVGURVFLZIPY2SKH73VRNZUNDZCO2J3BQMDCIHWALGKWG3NMJ7UA, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:1eKHcxWuBbQAGX44kVrn9cfEhEcRpmQfQegcoMGjr92pfM1VVF, address:104.36.82.54:4190): protocol: rpc: dial tcp 104.36.82.54:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682294511 (84h56m18.404857551s) [noise=true , address=104.36.82.53:4145 , piece_id=PBPYMEVYJMECJLDVDOMBLQQE4SG43K2EJT4ZIMHHMWA6LNQES4TQ, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:1rXBNYi6SgZdwGbiU5ooBE8U7DrEoAGw25ncYsRETgNh4CsiLX, address:104.36.82.53:4145): protocol: write tcp 188.166.225.170:34646->104.36.82.53:4145: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682334077 (84h56m18.404897117s) [noise=true , address=104.36.82.5:4850 , piece_id=GRA6TIREH63NIANSQABVYAWBLKNSIYO3I3NOQHO2OG6ZHMPEY5HA, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi, address:104.36.82.5:4850): protocol: rpc: dial tcp 104.36.82.5:4850: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682369875 (84h56m18.404932915s) [noise=true , address=104.36.82.13:4212 , piece_id=ABLX7JJ7GJUBUBRPRPWWSVSVMZQQDLHQVC4A5IBEEWBBHAL75SMA, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24, address:104.36.82.13:4212): protocol: rpc: dial tcp 104.36.82.13:4212: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682436473 (84h56m18.404999513s) [noise=true , address=104.36.82.10:4102 , piece_id=U6SX7XKPMSQLEFKXS4ULWXGPIXB3OZG3EMNHDHIWUOMXOMP2LNHA, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:1JEXRhcy1TphHqJXcd6qDv5gCdr6W2nJUcGoarZbsGt3TrtePy, address:104.36.82.10:4102): protocol: rpc: dial tcp 104.36.82.10:4102: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682464146 (84h56m18.405027186s) [noise=true , address=104.36.82.11:4116 , piece_id=BUBZZJU3M7VHXKVLSU65RP7LF4CR7U2ZNUV7I7NJ2DBIOAJX6I7Q, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:123aoo9g11aJP9hiL8shu2dx44tjyKzEWqPUjHUdXtR7LLFyng8, address:104.36.82.11:4116): protocol: rpc: dial tcp 104.36.82.11:4116: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682530663 (84h56m18.405093703s) [noise=true , address=104.36.82.21:4484 , piece_id=5NXUK4G5OLOOO2NW5XYYN7SES5C7HQ5CEFVBG6YS6GAG6JGPKURA, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:12dQKfD74Hp3MiUYqMm2oxGfHiStWTXdg4c4MEq8gCZ1hcbkgaN, address:104.36.82.21:4484): protocol: rpc: dial tcp 104.36.82.21:4484: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682561802 (84h56m18.405137076s) [noise=true , address=104.36.82.3:4774 , piece_id=6VDWT6HHR7NKG5SVO5FSJGB7L2WY7NXBGBNRTDKUGHYZSKSWMIZA, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:1r3Ty7Q8v3b4nT3cVVzQqGDFpHdBu1XKPQvBy221Y78hPzv4Y9, address:104.36.82.3:4774): protocol: rpc: dial tcp 104.36.82.3:4774: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682582472 (84h56m18.405145512s) [noise=true , address=104.36.82.50:4037 , piece_id=BEPAODIH6CG62BOTUCWMNWCHWAICSL4QNWSVIS6GKAXEWREQMBLQ, seg_pos={0 0}, upload=81489]: Done uploading piece. err: ecclient: upload failed (node:14gJ4XfdRiFLVEixdjSP5QB8F9toGcb6xoxP7kXxX1vXJm5TDg, address:104.36.82.50:4037): protocol: write tcp 188.166.225.170:44124->104.36.82.50:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:32:08.682605506 (84h56m18.405168546s) [seg_pos={0 0}, upload=81489]: Done waiting for segment. successful: 1 optimal: 14 errs: <nil> Jul 28 00:32:08.913172440 (84h56m18.6357355s) [upload=81489]: Done uploading segments... Jul 28 00:32:08.913202026 (84h56m18.635765066s) [upload=81489]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 1) Duration: 1m0.232795158s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................................X.......X.XX....X....X...] duration: 5.840965856s FAIL (2025-07-28 00:31:50 UTC) orbiter warning: digest mismatch: after 6b93ca9819ba9c70b065b54057b3f984abfbe2ed3f41c7f9ca09ec5e2ec8ba56 != before bfb4991c3c789f3b76f90db2ddfb43319ad13805292a8a70202450851dea6594 Duration: 5.840992387s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................X.X..........X.....X.X....] duration: 5.968349099s FAIL (2025-07-28 00:31:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.968374037s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................X.......X.XX....X....X..] duration: 5.78484549s FAIL (2025-07-28 00:31:17 UTC) orbiter warning: digest mismatch: after 6b93ca9819ba9c70b065b54057b3f984abfbe2ed3f41c7f9ca09ec5e2ec8ba56 != before bfb4991c3c789f3b76f90db2ddfb43319ad13805292a8a70202450851dea6594 Duration: 5.784876308s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 23 out of 60 is failed (threshold=3) [..................................X...XXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.237756454s FAIL (2025-07-28 00:31:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 1) ======LOGS====== L5GLKLHAWZ3CP34LAMXZ5HQ7VM5FJINWQEZHKH63XRA, seg_pos={0 0}, upload=81473]: Uploading piece... Jul 28 00:30:09.315880511 (84h54m19.038443561s) [noise=true , address=104.36.82.20:4472 , piece_id=3NT6GH43DWZ6QAROTKN4GUTKBU54C4JY27MAZMDFMRGIP3ZFFPNA, seg_pos={0 0}, upload=81473]: Uploading piece... Jul 28 00:30:09.315886262 (84h54m19.038449302s) [noise=true , address=104.36.82.42:4819 , piece_id=SPVP5ZWCMYPRJEOV7NYBEPZTOU2F3LFYNCI2LGVSMXFX5Y7L4BGQ, seg_pos={0 0}, upload=81473]: Uploading piece... Jul 28 00:30:09.316124338 (84h54m19.038687378s) [noise=true , address=104.36.82.12:4179 , piece_id=6ONZVCL5TM6XXM2AS2YRSONW5G2PZX27525I74KV5YDYNXDISQOA, seg_pos={0 0}, upload=81473]: Uploading piece... Jul 28 00:30:09.316408992 (84h54m19.038972032s) [noise=true , address=104.36.82.6:4880 , piece_id=LSOYO37RTEZLBG5VOGS2PLBBA6EVA5FFOBOMJCN6VVGHZVCZACHQ, seg_pos={0 0}, upload=81473]: Uploading piece... Jul 28 00:30:09.316705359 (84h54m19.039268399s) [noise=true , address=104.36.82.14:4245 , piece_id=K7PKDJSD6QAKNPSKTCM237HAT4O2ZASB4DONSRT2XYNXQ6BKJTDQ, seg_pos={0 0}, upload=81473]: Uploading piece... Jul 28 00:30:09.317028257 (84h54m19.039591297s) [noise=true , address=104.36.82.52:4108 , piece_id=TJZHDA6PJLBGKZ7Y45MX2H37UFUGP6EBBT2Z463LVIN5GEVGJD4A, seg_pos={0 0}, upload=81473]: Uploading piece... Jul 28 00:30:10.811127937 (84h54m20.533690977s) [upload=81473]: Next returned nil segment Jul 28 00:30:10.811154317 (84h54m20.533717357s) [upload=81473]: Waiting for error group managing segments... Jul 28 00:30:11.323185866 (84h54m21.045748906s) [noise=true , address=79.127.201.215:4354 , piece_id=AVOXGPFUBRFWOLPNHPHICKP5TIMVCBO3OVYGI2GEKWBZOS676NBQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: <nil> Jul 28 00:31:08.214913727 (84h55m17.937476778s) [noise=true , address=104.36.82.42:4819 , piece_id=SPVP5ZWCMYPRJEOV7NYBEPZTOU2F3LFYNCI2LGVSMXFX5Y7L4BGQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:12MZhrDu1fjMN7XFoR2h1q3JPLHJYYURGQwQgcgLuucSrGShjoS, address:104.36.82.42:4819): protocol: write tcp 188.166.225.170:50914->104.36.82.42:4819: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.214967641 (84h55m17.937530681s) [noise=true , address=104.36.82.4:4825 , piece_id=ZWHDXCROQL5GLKLHAWZ3CP34LAMXZ5HQ7VM5FJINWQEZHKH63XRA, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:104.36.82.4:4825): protocol: write tcp 188.166.225.170:54644->104.36.82.4:4825: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.214994943 (84h55m17.937557983s) [noise=true , address=104.36.82.12:4179 , piece_id=6ONZVCL5TM6XXM2AS2YRSONW5G2PZX27525I74KV5YDYNXDISQOA, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:129pP9f3rFfTvP6MhG894N7WHuJRZP8aWhzJf1zBwZ76wbzVbZa, address:104.36.82.12:4179): protocol: rpc: dial tcp 104.36.82.12:4179: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215011113 (84h55m17.937574153s) [noise=true , address=104.36.82.54:4174 , piece_id=4BLMWJZIUUDDMVULQWLZZVPZWNCZEVZAYZYDD3VK56E2IHV5E6KA, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:1293SNDGTKNrYrKDuB5p2VpHWo7ANpQsVKVGa97jXVaGP83Pw51, address:104.36.82.54:4174): protocol: write tcp 188.166.225.170:55018->104.36.82.54:4174: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215050108 (84h55m17.937613148s) [noise=true , address=104.36.82.46:4514 , piece_id=NFMU6NF43S2ST53R64WRIBTOXZLUWAKDCYP5GWGLI4YEULW5DSQQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:1iJgoxnaKnevARfWPAkd6ygyKXHZeVH8BvJPH13zbev6mGw2Bk, address:104.36.82.46:4514): protocol: write tcp 188.166.225.170:39740->104.36.82.46:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215062872 (84h55m17.937625912s) [noise=true , address=104.36.82.14:4245 , piece_id=K7PKDJSD6QAKNPSKTCM237HAT4O2ZASB4DONSRT2XYNXQ6BKJTDQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:1Zu8teNPTXn638grHKwtJqz4UEY7oFYLx1DHvuVP6RZXcatVqX, address:104.36.82.14:4245): protocol: rpc: dial tcp 104.36.82.14:4245: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215086518 (84h55m17.937649558s) [noise=true , address=104.36.82.20:4472 , piece_id=3NT6GH43DWZ6QAROTKN4GUTKBU54C4JY27MAZMDFMRGIP3ZFFPNA, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:12GACnovbrme3fQknSDpZWVLq838VKop9mPjxnVXGdJ8KaqyaTi, address:104.36.82.20:4472): protocol: write tcp 188.166.225.170:45758->104.36.82.20:4472: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215100725 (84h55m17.937663765s) [noise=true , address=104.36.82.9:4029 , piece_id=HMQQPRLIQR72MNJCFSVGPFKWHKAU2EZEK6XKOKPUJGBHW5KWVCZQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:12snsP1y5fbxY8XrDoXihzQrsBxJ5Y38C2jSGhHtV1ukN426eiP, address:104.36.82.9:4029): protocol: rpc: dial tcp 104.36.82.9:4029: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215125703 (84h55m17.937688743s) [noise=true , address=104.36.82.52:4108 , piece_id=TJZHDA6PJLBGKZ7Y45MX2H37UFUGP6EBBT2Z463LVIN5GEVGJD4A, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:12XJJZCUSHUS52YC6jhoPB4ayjsyKZT477PymPj4inkZtdUvheT, address:104.36.82.52:4108): protocol: write tcp 188.166.225.170:38876->104.36.82.52:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215153136 (84h55m17.937716176s) [noise=true , address=104.36.82.15:4260 , piece_id=2U7ZZJD5PJYJDFMCYGFKP4W2PIVTHBOAQYYOBMANKLM7PXDYXYRA, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:1aLsgrV3k6vUSavgPkPGA2FF9HR3ryN9bnHutruFJHW1GYixeL, address:104.36.82.15:4260): protocol: rpc: dial tcp 104.36.82.15:4260: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215164487 (84h55m17.937727527s) [noise=true , address=104.36.82.47:4532 , piece_id=MVPSNL2Z3Y6W5THFG5YCDJHNYQOAOEFEK7JC7HKGQRIK7UYXU4KQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL, address:104.36.82.47:4532): protocol: write tcp 188.166.225.170:47328->104.36.82.47:4532: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215200516 (84h55m17.937763556s) [noise=true , address=104.36.82.5:4846 , piece_id=YJHNLSHG4AKFEXTWSSGJUREJRHHGOPOMBXCDHDDKHPMP76ITUMNA, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:12MRTmrurAdfTtQqWboq2Z44yrzBKivkihfXeZ9bkCeGPcLF6iR, address:104.36.82.5:4846): protocol: write tcp 188.166.225.170:39548->104.36.82.5:4846: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215203912 (84h55m17.937766952s) [noise=true , address=104.36.82.11:4141 , piece_id=LKU2GDUWJ4DF24GUF7N5L5HGPOXZEMJR2FO7HFJKSYV3COYMGMPQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:129zfpWSguxcUH9723ivdoL39WgpjiTxA189mYbzsGAGVjFcXLL, address:104.36.82.11:4141): protocol: rpc: dial tcp 104.36.82.11:4141: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215231575 (84h55m17.937794615s) [noise=true , address=104.36.82.53:4145 , piece_id=BKBSWXXQIEH6GAO3PXKHTXKAP5RO4GI7VMWH2GHCQ3JLMTH6M2DQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:1rXBNYi6SgZdwGbiU5ooBE8U7DrEoAGw25ncYsRETgNh4CsiLX, address:104.36.82.53:4145): protocol: write tcp 188.166.225.170:34284->104.36.82.53:4145: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215244410 (84h55m17.93780745s) [noise=true , address=104.36.82.41:4781 , piece_id=56MTISJ7SHJMD6SUQZJPVIQCLQVJRUK4WKPDIIYVYPNPGK6X36BA, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:1QhCkTzSogXFKLmFV4wQUcALhvm9YhCSQudLxSrvQWmepheiFi, address:104.36.82.41:4781): protocol: rpc: dial tcp 104.36.82.41:4781: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215263938 (84h55m17.937826978s) [noise=true , address=104.36.82.40:4949 , piece_id=HN3YGGFI4SADN5VKNAJIOANBRB5NNZ6FHBNVUCTAURCX43H4V3QQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:12fLpEnaKeREB5itFjYy5yxy15a9GRT2w4p512qvB9wPTXZZwGo, address:104.36.82.40:4949): protocol: rpc: dial tcp 104.36.82.40:4949: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215299706 (84h55m17.937862736s) [noise=true , address=104.36.82.6:4880 , piece_id=LSOYO37RTEZLBG5VOGS2PLBBA6EVA5FFOBOMJCN6VVGHZVCZACHQ, seg_pos={0 0}, upload=81473]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: rpc: dial tcp 104.36.82.6:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:31:08.215327409 (84h55m17.937890439s) [seg_pos={0 0}, upload=81473]: Done waiting for segment. successful: 1 optimal: 14 errs: <nil> Jul 28 00:31:08.450124376 (84h55m18.172687416s) [upload=81473]: Done uploading segments... Jul 28 00:31:08.450153682 (84h55m18.172716722s) [upload=81473]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 1) Duration: 1m0.237771473s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................X............X.......................X..] duration: 2.033520161s FAIL (2025-07-28 00:30:54 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 2.033546382s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................X.......X.XX....X....X.] duration: 5.902269129s FAIL (2025-07-28 00:30:46 UTC) orbiter warning: digest mismatch: after 6b93ca9819ba9c70b065b54057b3f984abfbe2ed3f41c7f9ca09ec5e2ec8ba56 != before bfb4991c3c789f3b76f90db2ddfb43319ad13805292a8a70202450851dea6594 Duration: 5.902301412s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................................X.X..........X.....X.X...] duration: 5.769486748s FAIL (2025-07-28 00:30:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.769518318s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................................X.X..........X.....X.X..] duration: 6.167570351s FAIL (2025-07-28 00:30:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.167598365s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [...................................X...XXXXXXXXXXXXXXXXXXXXX] duration: 1m0.247360731s FAIL (2025-07-28 00:30:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 2) ======LOGS====== ise=true , address=104.36.82.46:4493 , piece_id=GZTBIMVTJOQWGFRAKQVEVE7H34K2GEQXLDC4LHM2XKVCIL7EQRIQ, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:08.887441785 (84h53m18.610004865s) [noise=true , address=104.36.82.54:4163 , piece_id=RWP2HS5JELWA3GYEPACV3XGK7LFMGKXZUDUMGIG2QHY2PQYS5YTA, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:08.887689739 (84h53m18.610252779s) [noise=true , address=104.36.82.43:4856 , piece_id=33SNVGLP2XTI7D6LBRSGYEBKLTLAYFDTIP3VFSKGGQ73QVU6L3ZA, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:08.887883930 (84h53m18.610446971s) [noise=true , address=104.36.82.7:4918 , piece_id=BVKNRJGYE3YGHY5M52Q6FWL5K4P7FU6N2UODEROC5LQG65QWMJMQ, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:08.888077161 (84h53m18.610640201s) [noise=true , address=104.36.82.3:4778 , piece_id=MKNARU5QZTMG625B4GFW4J4FNOLIZURB2HLL6GIOMDM6GAQTIX5Q, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:08.888259018 (84h53m18.610822059s) [noise=true , address=104.36.82.20:4464 , piece_id=IUT7RSUTOAUMW2BT4IN7LCYJTWPTQJVOHPA6QY2KKSKF4QZZYFCQ, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:08.888535427 (84h53m18.611098467s) [noise=true , address=104.36.82.22:4531 , piece_id=ZP3VXKPPV7F5LHDOK7WP6WG5OLYB2OEATDOIS66EGEILSPEKT4FA, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:08.888817667 (84h53m18.611380707s) [noise=true , address=104.36.82.14:4253 , piece_id=TDPWZKXMJBJAO362ABF6M2LVRCVVLDXXBELY2HAF27ANJWKQKFRQ, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:08.889002441 (84h53m18.611565481s) [noise=true , address=104.36.82.4:4836 , piece_id=BFT53MZPK5UKFOK7EV5ZLXU3PBLG6CYR5AUSTGFNW7WLMQGMKTPA, seg_pos={0 0}, upload=81459]: Uploading piece... Jul 28 00:29:10.028440866 (84h53m19.751003906s) [upload=81459]: Next returned nil segment Jul 28 00:29:10.028461976 (84h53m19.751025026s) [upload=81459]: Waiting for error group managing segments... Jul 28 00:29:10.317759258 (84h53m20.040322308s) [noise=true , address=79.127.205.233:4656 , piece_id=6JFAP3LPIDO6OQXRRUERLLFXJONIQLBE46NKC6UH4NRKTBN3IJKA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: <nil> Jul 28 00:29:14.415848674 (84h53m24.138411714s) [noise=true , address=79.127.201.215:4350 , piece_id=IZS64LLQDU34BQTMIY7X4HR4FKW5X5TMRAOXSQ4W3GBZNMOB74AQ, seg_pos={0 0}, upload=81459]: Done uploading piece. err: <nil> Jul 28 00:30:07.739033102 (84h54m17.461596142s) [noise=true , address=104.36.82.54:4163 , piece_id=RWP2HS5JELWA3GYEPACV3XGK7LFMGKXZUDUMGIG2QHY2PQYS5YTA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12CgPQ5hkrJdqbJvGarxKjXvYYnrXQqapL5BH2rNJNvnhpRi3pw, address:104.36.82.54:4163): protocol: write tcp 188.166.225.170:50186->104.36.82.54:4163: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739408259 (84h54m17.461971299s) [noise=true , address=104.36.82.22:4531 , piece_id=ZP3VXKPPV7F5LHDOK7WP6WG5OLYB2OEATDOIS66EGEILSPEKT4FA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12jGenn3Fh2VB36jVPzyy2LxJgbLraXxL2yobw9tsBJLSDPJycN, address:104.36.82.22:4531): protocol: rpc: dial tcp 104.36.82.22:4531: i/o timeout; conn closed: op Jul 28 00:30:07.739451262 (84h54m17.462014302s) [noise=true , address=104.36.82.46:4493 , piece_id=GZTBIMVTJOQWGFRAKQVEVE7H34K2GEQXLDC4LHM2XKVCIL7EQRIQ, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:127drPYZWm7RRXuCLtuzDgSaxxuu3k1tg6eo3KWtrQGjBaUbfRJ, address:104.36.82.46:4493): protocol: write tcp 188.166.225.170:59864->104.36.82.46:4493: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739489314 (84h54m17.462052354s) [noise=true , address=104.36.82.52:4103 , piece_id=P4ZFDPEO6SGYRVKVNDINKSRNMEUMTSOTOFBAYVLBQOC3KSBWYXAA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:1v6zXZuMmyL2MU8qb3q6gg1iwTYvndjzMjVpv8btEpeX9Lnqmu, address:104.36.82.52:4103): protocol: rpc: dial tcp 104.36.82.52:4103: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739535352 (84h54m17.462098393s) [noise=true , address=104.36.82.40:4950 , piece_id=ANGPYTWM4DOD7CEE63BFI2RHWQEI5KJJHCDUSEFQAHJOPEICY4WQ, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:1Y6fZC1qCRQYMEvW3475SvnaWYCidxcDhch2YA1ZXW5U8fXDZk, address:104.36.82.40:4950): protocol: rpc: dial tcp 104.36.82.40:4950: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739565450 (84h54m17.46212849s) [noise=true , address=104.36.82.3:4778 , piece_id=MKNARU5QZTMG625B4GFW4J4FNOLIZURB2HLL6GIOMDM6GAQTIX5Q, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12JUnoFQa1s1amHUHWiTtQVatCzgMHii1kskRjm9DYjkEuke3Ge, address:104.36.82.3:4778): protocol: rpc: dial tcp 104.36.82.3:4778: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739613392 (84h54m17.462176432s) [noise=true , address=104.36.82.4:4836 , piece_id=BFT53MZPK5UKFOK7EV5ZLXU3PBLG6CYR5AUSTGFNW7WLMQGMKTPA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:104.36.82.4:4836): protocol: rpc: dial tcp 104.36.82.4:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739646585 (84h54m17.462209625s) [noise=true , address=104.36.82.53:4147 , piece_id=BVGHLS63MBLO2EMSNIFICVH6LY7GXOEDE7G4JE4GY5E7Y5EXEWPQ, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12pg6fGWqRjS8Gf3zQgw8jbuytaCyMpJ2CAz4oRVHYdLAhttgnH, address:104.36.82.53:4147): protocol: write tcp 188.166.225.170:34608->104.36.82.53:4147: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739669419 (84h54m17.462232459s) [noise=true , address=104.36.82.42:4828 , piece_id=6WPO7MGZONWFDKUEFVLWNXVE4SWJ7B6GXZBMRXYIDZJT5BOF4IAA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:104.36.82.42:4828): protocol: write tcp 188.166.225.170:60528->104.36.82.42:4828: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739715738 (84h54m17.462278778s) [noise=true , address=104.36.82.56:4261 , piece_id=3CYQYT4IOKLOOZCIPWINDAQWB7MDM5VVSGAUJ76OB46KC3DYPUPQ, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12YNKaVEsXzHe6XGQuv6wR1qqbv6dbzg1xT5Aqkew2ZV3z3tMMf, address:104.36.82.56:4261): protocol: rpc: dial tcp 104.36.82.56:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739759511 (84h54m17.462322551s) [noise=true , address=104.36.82.44:4872 , piece_id=FEZYYYVMNTD73OIB5OWIULSMIHUIRU3L7QQ6OIYCL5DZNDYJJQEA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12dNdm2kAok8jFgbYE24oXUL9KCJVDDiinLUJgDSYEY3fTEBJec, address:104.36.82.44:4872): protocol: rpc: dial tcp 104.36.82.44:4872: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739795361 (84h54m17.462358401s) [noise=true , address=104.36.82.7:4918 , piece_id=BVKNRJGYE3YGHY5M52Q6FWL5K4P7FU6N2UODEROC5LQG65QWMJMQ, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12EMGZ4hpXmBp4TwemLrbKAVtjBmtexMzEahRTzFw42qQipsAG, address:104.36.82.7:4918): protocol: rpc: dial tcp 104.36.82.7:4918: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739827522 (84h54m17.462390562s) [noise=true , address=104.36.82.20:4464 , piece_id=IUT7RSUTOAUMW2BT4IN7LCYJTWPTQJVOHPA6QY2KKSKF4QZZYFCQ, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:1xKikZTtJ6dr1ry8oAkLc7fmyB51jydk87BWoxuTTzKFDeToEY, address:104.36.82.20:4464): protocol: rpc: dial tcp 104.36.82.20:4464: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739834094 (84h54m17.462397134s) [noise=true , address=104.36.82.14:4253 , piece_id=TDPWZKXMJBJAO362ABF6M2LVRCVVLDXXBELY2HAF27ANJWKQKFRQ, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12vJmwywMNTHh1ce5JGzbMiUBA3FijDecNEJ1TDU1Wyd4NWzA14, address:104.36.82.14:4253): protocol: rpc: dial tcp 104.36.82.14:4253: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739861156 (84h54m17.462424196s) [noise=true , address=104.36.82.55:4207 , piece_id=PVVAD3TFTA6TAHCBQQN4GT3HPBVJEMGOCJKRXOMG2KKLH3AJ2DEA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:121hXqdHirzNQq5H1ynCC3bboxCyYpeRfBbZR4zPQCPiW2ebbey, address:104.36.82.55:4207): protocol: rpc: dial tcp 104.36.82.55:4207: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739883058 (84h54m17.462446098s) [noise=true , address=104.36.82.43:4856 , piece_id=33SNVGLP2XTI7D6LBRSGYEBKLTLAYFDTIP3VFSKGGQ73QVU6L3ZA, seg_pos={0 0}, upload=81459]: Done uploading piece. err: ecclient: upload failed (node:12bQxwBfPEGtmMp6mtnWKsoZt3eMEjLdyYFahETWGHeKJ7BhkiZ, address:104.36.82.43:4856): protocol: rpc: dial tcp 104.36.82.43:4856: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:30:07.739908477 (84h54m17.462471517s) [seg_pos={0 0}, upload=81459]: Done waiting for segment. successful: 2 optimal: 14 errs: <nil> Jul 28 00:30:07.984812080 (84h54m17.70737513s) [upload=81459]: Done uploading segments... Jul 28 00:30:07.984846245 (84h54m17.707409285s) [upload=81459]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 2) Duration: 1m0.247388394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................X............X.......................X.] duration: 1.905960721s FAIL (2025-07-28 00:30:04 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.905999696s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................X.......X.XX....X....X] duration: 5.634699794s FAIL (2025-07-28 00:29:45 UTC) orbiter warning: digest mismatch: after 6b93ca9819ba9c70b065b54057b3f984abfbe2ed3f41c7f9ca09ec5e2ec8ba56 != before bfb4991c3c789f3b76f90db2ddfb43319ad13805292a8a70202450851dea6594 Duration: 5.634725233s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................................X.X..........X.....X.X.] duration: 5.732139311s FAIL (2025-07-28 00:29:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.732166926s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................X.X..........X.....X.X] duration: 30.236993658s FAIL (2025-07-28 00:29:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== : piecestore close: conn closed: op Jul 28 00:28:35.518933456 (84h52m45.241496486s) [noise=true , address=77.90.31.14:4663 , piece_id=D4WW6RGHTKEAO2T5WKRIE7WFK2LSOH3A2KYXPRQROSH2SHCK5LWA, seg_pos={0 0}, upload=81449]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Jul 28 00:28:35.518966329 (84h52m45.241529369s) [seg_pos={0 0}, upload=81449]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:28:35.519050750 (84h52m45.24161379s) [upload=81449]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:28:35.519055230 (84h52m45.24161827s) [upload=81449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:28:35.809458868 (84h52m45.532021898s) [upload=81449]: Done uploading segments... Jul 28 00:28:35.809476372 (84h52m45.532039412s) [upload=81449]: Upload finished. err: <nil> Jul 28 00:29:02.559852893 (84h53m12.282415933s) [upload=81457]: Starting upload Jul 28 00:29:02.559924630 (84h53m12.282487671s) [upload=81457]: Done starting upload Jul 28 00:29:02.560098182 (84h53m12.282661222s) [upload=81457]: Uploading segments... Jul 28 00:29:02.560181612 (84h53m12.282744652s) [upload=81457]: Got next segment. Inline: false Jul 28 00:29:02.560221077 (84h53m12.282784117s) [upload=81457]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:29:02.560225937 (84h53m12.282788977s) [upload=81457]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:29:03.009059749 (84h53m12.731622819s) [seg_pos={0 0}, upload=81457]: Begin upload segment... Jul 28 00:29:03.009158979 (84h53m12.73172206s) [seg_pos={0 0}, upload=81457]: Done begin upload segment. Jul 28 00:29:03.009178718 (84h53m12.731741788s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009188606 (84h53m12.731751656s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009792902 (84h53m12.732355942s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.009842818 (84h53m12.732405858s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010397319 (84h53m12.732960359s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010459187 (84h53m12.733022227s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010730286 (84h53m12.733293326s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.010957360 (84h53m12.7335204s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011154638 (84h53m12.733717688s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011401190 (84h53m12.73396425s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.011975508 (84h53m12.734538568s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.012958940 (84h53m12.73552198s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013411306 (84h53m12.735974346s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.013856397 (84h53m12.736419437s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.015192333 (84h53m12.737755373s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:03.016283781 (84h53m12.738846841s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Uploading piece... Jul 28 00:29:04.783335657 (84h53m14.505898687s) [upload=81457]: Next returned nil segment Jul 28 00:29:04.783353261 (84h53m14.505916301s) [upload=81457]: Waiting for error group managing segments... Jul 28 00:29:05.060157571 (84h53m14.782720611s) [noise=true , address=45.91.116.12:4226 , piece_id=IVJCX4DRW5MNEPRUJ3MZZI5FVH5VPN2MRMX4242LF5EXADY642GQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.099038051 (84h53m14.821601091s) [noise=true , address=77.90.31.14:4662 , piece_id=M3UZIFESWBE7ICRMSUG6AHQP4OKB4IB2CXIJASIATTWL3HAR5XCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.113329355 (84h53m14.835892395s) [noise=true , address=45.91.116.6:4974 , piece_id=JRKWVG4KHMZPP36IWHCZNS7RRRA3QA3KPGE2PP47BTHMIHBXBLEQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.116510420 (84h53m14.83907345s) [noise=true , address=45.91.116.14:4191 , piece_id=WVDBAGF7VY5GUF6JEOYBNJCQCMNNS2XXIQ3LBYY4LJ3BZ2J2SD7Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131296190 (84h53m14.85385923s) [noise=true , address=77.90.31.10:4520 , piece_id=7KTSJ5NMZCF6HKXVWBON4E6ADIE4SZYH6YJRRF5E66OG3ZZ6PPDQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.131761601 (84h53m14.854324641s) [noise=true , address=77.90.31.12:4590 , piece_id=AXYXD45UZXCL6DUJFUSQWDGK7YL5TNX2L75PCQMLD4TOB7GIETBA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.135486167 (84h53m14.858049197s) [noise=true , address=45.91.116.8:4045 , piece_id=DJQ6G5N6QOF23KJWJ5QG4VQULDK5FMHHW2YOW4CNN5WGGA23BJCQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.137908521 (84h53m14.860471561s) [noise=true , address=45.91.116.10:4082 , piece_id=ET7H6GHFO7TUCBHG2ZJGENBBTJXFACXJCZYEVM666GPSZHWQAL2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.141090389 (84h53m14.863653429s) [noise=true , address=77.90.31.12:4592 , piece_id=VVAGFRV7K23FE2IEUANVG2CXEMB3UPHO7QF6HVY4W7HEDUUXXDYA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.146438030 (84h53m14.869001071s) [noise=true , address=45.91.116.9:4263 , piece_id=KZ6CUSPAQF32EE5V7IBNZPXHZVYUI6E7JUIILPTTB5XQD4IHYO2Q, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.210337970 (84h53m14.93290101s) [noise=true , address=45.91.116.13:4155 , piece_id=OU4L4PVBLIIUJ3622P3YJVA2XBJOGB3IFTW7FQ66BUJXGXW7ALRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:05.271828060 (84h53m14.9943911s) [noise=true , address=77.90.31.18:4806 , piece_id=EDYSRWSRO227H4NZPNGBWFCC7XETV4MCNDVY4MBWH4UFUPAJ7QJA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: <nil> Jul 28 00:29:31.908491111 (84h53m41.631054151s) [noise=true , address=104.36.82.11:4113 , piece_id=D5YGTYWGJOOT3WNJY2MEOVKL25ONCOSTI7KDJLXVDBOAHZLAQBKQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908513103 (84h53m41.631076144s) [noise=true , address=104.36.82.15:4258 , piece_id=RNAF33EEEK4KICEEW66ZUGFVRKEL2RAW3EH3ARKXT3PDO5IEZULA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908534383 (84h53m41.631097423s) [noise=true , address=104.36.82.7:4911 , piece_id=AFGDPAP7IIQJ6PKZ34KXHG2GOUWQ7AIM3E6NQ4O56Z7QOGNUWLRQ, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:38878->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:31.908598877 (84h53m41.631161918s) [noise=true , address=104.36.82.9:4005 , piece_id=GI5GLJGPEXKPGJCU67P5QIJ5Q2OR5QVG2RGB7LQSWCWKLTOHPKRA, seg_pos={0 0}, upload=81457]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 28 00:29:31.908613525 (84h53m41.631176565s) [seg_pos={0 0}, upload=81457]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:29:32.144150239 (84h53m41.866713319s) [upload=81457]: Done uploading segments... Jul 28 00:29:32.144183573 (84h53m41.866746653s) [upload=81457]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.237015551s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................X............X.......................X] duration: 1.606808783s FAIL (2025-07-28 00:29:02 UTC) orbiter warning: digest mismatch: after f108761aa7d0920bb363621c2197fd3578093c870514492788074b7848c554e0 != before 8e770a63b4165ee9ca5f7101d5bcd1d72eb0c5d764fc243e1413ffe36e2d1ae6 Duration: 1.606830686s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [....................................X...XXXXXXXXXXXXXXXXXXXX] duration: 1m0.234042979s FAIL (2025-07-28 00:29:00 UTC) orbiter warning: ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0) ======LOGS====== load=81442]: Uploading piece... Jul 28 00:28:01.715304911 (84h52m11.437867951s) [noise=true , address=104.36.82.42:4820 , piece_id=CCFMFB42I6ZBJUJEJCDTKOSS5VB6IEJTNGIDZAVLCLLGQZFKLEKA, seg_pos={0 0}, upload=81442]: Uploading piece... Jul 28 00:28:01.715487180 (84h52m11.43805023s) [noise=true , address=104.36.82.52:4101 , piece_id=QKMOSYYUS2QQHXB5NBAI4K33XMJSRQBW4R6NLW2BLLPJU2ORFZWQ, seg_pos={0 0}, upload=81442]: Uploading piece... Jul 28 00:28:01.715521917 (84h52m11.438084957s) [noise=true , address=104.36.82.56:4252 , piece_id=FC4W55PLKPZKNDRSPDOAIX3MX7KEHHUWRJ6HCYIVD7HLAPJMWYQQ, seg_pos={0 0}, upload=81442]: Uploading piece... Jul 28 00:28:01.715743971 (84h52m11.438307011s) [noise=true , address=104.36.82.42:4815 , piece_id=JNOLUULYTG4ZVZ5AGWPBHQ5JYZGL6APUYLO2Q2TWOAWKP6FN5HQQ, seg_pos={0 0}, upload=81442]: Uploading piece... Jul 28 00:28:01.715752237 (84h52m11.438315277s) [noise=true , address=104.36.82.41:4774 , piece_id=YDBCHOPRX4AQMMISLUTXTEMWPYD7GCZ4XAGZO3DKW3HWEDDE34XQ, seg_pos={0 0}, upload=81442]: Uploading piece... Jul 28 00:28:01.715963111 (84h52m11.438526151s) [noise=true , address=104.36.82.3:4781 , piece_id=I2IS6MHRJAGH4WO2IDIESYIWODZKJAUUS6FAZMQ4NIMNYDI4RSWQ, seg_pos={0 0}, upload=81442]: Uploading piece... Jul 28 00:28:01.716050508 (84h52m11.438613558s) [noise=true , address=104.36.82.47:4532 , piece_id=FOHFBJ3JCJ5CKSZ74OOCT24BD7CHCDOE65HJNQWVHFUHBS2ETSIQ, seg_pos={0 0}, upload=81442]: Uploading piece... Jul 28 00:28:01.716310566 (84h52m11.438873616s) [noise=true , address=104.36.82.11:4139 , piece_id=VF7GOUAAQVAUYPWGJEDNDUBMCSZPW6Z6MBRM3YKZGD2PWN2C6UOA, seg_pos={0 0}, upload=81442]: Uploading piece... Jul 28 00:29:00.547854540 (84h53m10.27041758s) [upload=81442]: Next segment err: context deadline exceeded Jul 28 00:29:00.549016414 (84h53m10.271579454s) [noise=true , address=104.36.82.42:4815 , piece_id=JNOLUULYTG4ZVZ5AGWPBHQ5JYZGL6APUYLO2Q2TWOAWKP6FN5HQQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1F59CEsZMSXL8DibXEqG6XQKirqjEfYW8UhugnFCPnUEUWLHDX, address:104.36.82.42:4815): protocol: rpc: dial tcp 104.36.82.42:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549044748 (84h53m10.271607788s) [noise=true , address=104.36.82.14:4251 , piece_id=PYAOCLZIAZKDYKFOA52AKLVW7KFI224A2CS5WAIQH7LLLRRANZGQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:12btf6Z38aTMx7MaBxhUyZtFqUPgeMtP2aU8Y6FbV8zwu57zArd, address:104.36.82.14:4251): protocol: rpc: dial tcp 104.36.82.14:4251: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549074795 (84h53m10.271637835s) [noise=true , address=104.36.82.41:4774 , piece_id=YDBCHOPRX4AQMMISLUTXTEMWPYD7GCZ4XAGZO3DKW3HWEDDE34XQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1oUUQD6GJU2WHCYnyp98du4BG8wAXMvsoLvvaFfuAsfqA6mCfb, address:104.36.82.41:4774): protocol: rpc: dial tcp 104.36.82.41:4774: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549116354 (84h53m10.271679394s) [noise=true , address=104.36.82.56:4252 , piece_id=FC4W55PLKPZKNDRSPDOAIX3MX7KEHHUWRJ6HCYIVD7HLAPJMWYQQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1UDFNiqXb4riRuewJDPrKFykVWLiryNKqgkH6uCZszz6Sr8naB, address:104.36.82.56:4252): protocol: rpc: dial tcp 104.36.82.56:4252: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549131865 (84h53m10.271694905s) [noise=true , address=104.36.82.3:4781 , piece_id=I2IS6MHRJAGH4WO2IDIESYIWODZKJAUUS6FAZMQ4NIMNYDI4RSWQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1Y6iagayTdoSQtXrAdeBrvrck8qMNRAmLogi5jUG8bcLrBEswn, address:104.36.82.3:4781): protocol: rpc: dial tcp 104.36.82.3:4781: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549155440 (84h53m10.27171848s) [noise=true , address=104.36.82.20:4458 , piece_id=MEKHH7JFY4NXFP4OUQJECB5HS3KRUN4SAQ43LPKGABYPCII5DSOA, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:12SEsKJVwuQV1Z6s6doBrqsST1KhqbBREjwFYeEpME7gHExaG5d, address:104.36.82.20:4458): protocol: rpc: dial tcp 104.36.82.20:4458: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549208071 (84h53m10.271771111s) [noise=true , address=104.36.82.11:4139 , piece_id=VF7GOUAAQVAUYPWGJEDNDUBMCSZPW6Z6MBRM3YKZGD2PWN2C6UOA, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1EiZjV2QMYue3KQDsNRhhdGpCQjybDv54wTuXsJTNtyeV8hHeh, address:104.36.82.11:4139): protocol: rpc: dial tcp 104.36.82.11:4139: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549216937 (84h53m10.271779967s) [noise=true , address=104.36.82.46:4515 , piece_id=PGGPFVZHMOMBHP7CHPREHQJVUZ66RMELJBFFXBQYRZAMCGF7M5UA, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1iDufTPd97E4DAupki6D6DxNZS8S1gKwynNBYynsauhbw5waUs, address:104.36.82.46:4515): protocol: rpc: dial tcp 104.36.82.46:4515: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549247335 (84h53m10.271810375s) [noise=true , address=104.36.82.44:4893 , piece_id=H6W74NMNU427ENMB5U46FUHEEMQD2DS4QZLORGWHEYMZYHADDAUA, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:17VMFZ2T8jrSJVtchciJ9SnyrR6JstvcMoXrN5Q3zTD1aQU1oJ, address:104.36.82.44:4893): protocol: rpc: dial tcp 104.36.82.44:4893: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549285458 (84h53m10.271848498s) [noise=true , address=104.36.82.21:4477 , piece_id=MA6PTPUA3RHZAWNKJNFYG77D7Y4IJTK7YQPFNND7SAMQNQIHHRZA, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1QNztqPaSGyZVsQihaj9YjG9C1nzKi1crfSDjDHimYboYAKWZY, address:104.36.82.21:4477): protocol: rpc: dial tcp 104.36.82.21:4477: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549315335 (84h53m10.271878375s) [noise=true , address=104.36.82.51:4079 , piece_id=G6YM5XVCZTO2XNZWFAPQPVKPVLDXQ4F7SDNH5Y6Y635PFK5F2JIQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:12RYKzn5zVSmxVbWwnBdX8oqtG1WE2MhPeEiWLf5dZeFAut6s2Y, address:104.36.82.51:4079): protocol: rpc: dial tcp 104.36.82.51:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549329392 (84h53m10.271892432s) [noise=true , address=104.36.82.52:4101 , piece_id=QKMOSYYUS2QQHXB5NBAI4K33XMJSRQBW4R6NLW2BLLPJU2ORFZWQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:12WWXeBjFZR9PzciNfs35fFcx87N9GwDpMsPHnksFDugtaBGWp4, address:104.36.82.52:4101): protocol: rpc: dial tcp 104.36.82.52:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549368747 (84h53m10.271931787s) [noise=true , address=104.36.82.42:4820 , piece_id=CCFMFB42I6ZBJUJEJCDTKOSS5VB6IEJTNGIDZAVLCLLGQZFKLEKA, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:12rWzLpDf6jtpMcAFyF4WWUgcWV6cJMKNWRo6wZYvtZ1rDoQ1Hh, address:104.36.82.42:4820): protocol: rpc: dial tcp 104.36.82.42:4820: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549382744 (84h53m10.271945774s) [noise=true , address=104.36.82.50:4045 , piece_id=RZHH5RSDBNSCHZB7ZLG7SXLPMEXWZM5KR4X6JUCK4TOSNRTO6GVQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1j6J2T4aX3bmjtP9J379mNQBFDp3vQZeAvmUFEYVfYZFknjrTp, address:104.36.82.50:4045): protocol: rpc: dial tcp 104.36.82.50:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549415717 (84h53m10.271978757s) [noise=true , address=104.36.82.54:4188 , piece_id=VLHOGJFCZVI7ZY7JQEPW2S2EFBZVCZEZPEIMFL4UP3R62PZDQD4Q, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1HQ41iFa5jBBMhjL3LArqNfVhmKzNzF4N436n4qdod6GoNhAQG, address:104.36.82.54:4188): protocol: write tcp 188.166.225.170:54936->104.36.82.54:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549468719 (84h53m10.272031749s) [noise=true , address=104.36.82.10:4089 , piece_id=OLEQUNY4QRABHKMACDVA5UTIIDOQDINC2CSBR7MP5724LGQVD4NQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:12bFi2opJ6FejAMwSL2qbqWoCk24QzSjT6xQYE8BytLgVJikZMT, address:104.36.82.10:4089): protocol: write tcp 188.166.225.170:38556->104.36.82.10:4089: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549477255 (84h53m10.272040295s) [noise=true , address=104.36.82.47:4532 , piece_id=FOHFBJ3JCJ5CKSZ74OOCT24BD7CHCDOE65HJNQWVHFUHBS2ETSIQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:15buPXDh9xXPzcL5Z7q2XZTQujajVRBp97VnPp5ngUDQL26AbL, address:104.36.82.47:4532): protocol: write tcp 188.166.225.170:47522->104.36.82.47:4532: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549485992 (84h53m10.272049032s) [noise=true , address=104.36.82.53:4137 , piece_id=FCWMACIPJZAODEWSE6DOAH3XFKRJNRXVE6OQZFHP557JEQPOLJWQ, seg_pos={0 0}, upload=81442]: Done uploading piece. err: ecclient: upload failed (node:1dZZBaTDA1kbdJmNZzzYgdPkKE2rNvEqdWaMGUxuTD2ERbSanc, address:104.36.82.53:4137): protocol: rpc: dial tcp 104.36.82.53:4137: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:29:00.549496772 (84h53m10.272059812s) [seg_pos={0 0}, upload=81442]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil> Duration: 1m0.234058308s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................X.X..........X.....X.] duration: 6.007484627s FAIL (2025-07-28 00:28:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 81424]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: rpc: dial tcp 104.36.82.6:4874: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:27:00.887031015 (84h51m10.609594045s) [seg_pos={0 0}, upload=81424]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 28 00:27:00.887091471 (84h51m10.609654511s) [upload=81424]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 28 00:27:00.887096300 (84h51m10.60965934s) [upload=81424]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 28 00:27:01.163946963 (84h51m10.886510003s) [upload=81424]: Done uploading segments... Jul 28 00:27:01.163965929 (84h51m10.886528969s) [upload=81424]: Upload finished. err: <nil> Jul 28 00:27:19.597171719 (84h51m29.319734759s) [upload=81431]: Starting upload Jul 28 00:27:19.597272533 (84h51m29.319835573s) [upload=81431]: Done starting upload Jul 28 00:27:19.597365821 (84h51m29.319928861s) [upload=81431]: Uploading segments... Jul 28 00:27:19.597700883 (84h51m29.320263923s) [upload=81431]: Got next segment. Inline: false Jul 28 00:27:19.597736120 (84h51m29.32029915s) [upload=81431]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 28 00:27:19.597741259 (84h51m29.320304299s) [upload=81431]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 28 00:27:20.045867384 (84h51m29.768430424s) [seg_pos={0 0}, upload=81431]: Begin upload segment... Jul 28 00:27:20.045927990 (84h51m29.76849103s) [seg_pos={0 0}, upload=81431]: Done begin upload segment. Jul 28 00:27:20.045958679 (84h51m29.768521729s) [noise=true , address=77.90.31.15:4698 , piece_id=X5L3W67HMYGRJVZYU72WUJ2ZT3VZ23DRBQJCNUJUV6YZCOL5ICTQ, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.045965942 (84h51m29.768528982s) [noise=true , address=77.90.31.13:4627 , piece_id=7NR6V4GBJR5WOHQRPDO4VIWT5BPEHWJMRE36UHKIIWSI6VQE6TAA, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.046055715 (84h51m29.768618755s) [noise=true , address=45.91.116.8:4047 , piece_id=GEDNNAD5CKPNTE7I2X2IK3SCI2G2F3JPKEVXW3HLIO44XOFKIURA, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.046336261 (84h51m29.768899302s) [noise=true , address=77.90.31.10:4519 , piece_id=N5HFTGGOKYCT42TILLRRHTJRYH4ZRWDQ32ADKNPGJN3ND2NQSUMQ, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.046371579 (84h51m29.768934619s) [noise=true , address=45.91.116.9:4262 , piece_id=6GCZ7XGI4ZRYXBSN3MKUXSOKDFBBMKWWOXDU4VVU56ZDILYUHDEA, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.046630915 (84h51m29.769193955s) [noise=true , address=45.91.116.7:4010 , piece_id=FQ35UKX2QBOJ7KNNL44QQPNJWYOPBVUKNC67BTPFVQGJ2YUM2CSA, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.046655041 (84h51m29.769218081s) [noise=true , address=104.36.82.2:4731 , piece_id=KO3XYSDCEOM2YFS3LJOF27UIUULA4ZVRUXFWGPJYR5RJR3BCK45Q, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.046905961 (84h51m29.769469001s) [noise=true , address=45.91.116.11:4118 , piece_id=5OBIONSVURIRITMD2HAVHLZTIDIBQOZJENJPKAI5JVKX6NSVG4ZQ, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.046926390 (84h51m29.76948943s) [noise=true , address=77.90.31.16:4734 , piece_id=LDBACEVIC452LRFQ4NZWTWVJDIQZWD6HJL4XCDS55CJ362LACQBA, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.047221144 (84h51m29.769784184s) [noise=true , address=104.36.82.10:4077 , piece_id=RTY63EEKUWRHDL6S5WUDY5HAGUFEGFR5BWZO2QAKCBD7UC2JCQRQ, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.047224911 (84h51m29.769787951s) [noise=true , address=104.36.82.4:4802 , piece_id=FFNXZCMDTS632HPHFDWEOXRR3XIMFDG2UEYOAXFFESCQJIWBRWUA, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.047435665 (84h51m29.769998705s) [noise=true , address=77.90.31.19:4842 , piece_id=SOQDD53C55RO67Q2C5HAYFTSKZP7ZMEWU42OON3YW24MQUMFNJ3A, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.047460853 (84h51m29.770023893s) [noise=true , address=77.90.31.11:4555 , piece_id=SZQIQFGZUB225EUHDDT4JMJYMRIJ6FE25BXHM4ZGPDJM4SGTBPQQ, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.047706483 (84h51m29.770269533s) [noise=true , address=77.90.31.21:4879 , piece_id=QBLWJK4D3WXQ4CO7U2XPX262ZDFPLLQKWUUUD43O5BHBCSBZBJLA, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.047961571 (84h51m29.770524611s) [noise=true , address=77.90.31.14:4662 , piece_id=PUBSU5WGV2VSPXER4D255XNPMZPBTLOLA5JS5ZBJFJWKWA6CYFBA, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:20.048175260 (84h51m29.7707383s) [noise=true , address=104.36.82.6:4876 , piece_id=3HCF5CQHV7P73Z5I5NVFRUVGPBT5SCU24QXZGOZHRYG3MSA5HOUQ, seg_pos={0 0}, upload=81431]: Uploading piece... Jul 28 00:27:21.770812904 (84h51m31.493375944s) [upload=81431]: Next returned nil segment Jul 28 00:27:21.770831569 (84h51m31.493394609s) [upload=81431]: Waiting for error group managing segments... Jul 28 00:27:22.047721888 (84h51m31.770284928s) [noise=true , address=45.91.116.9:4262 , piece_id=6GCZ7XGI4ZRYXBSN3MKUXSOKDFBBMKWWOXDU4VVU56ZDILYUHDEA, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.069919327 (84h51m31.792482367s) [noise=true , address=77.90.31.14:4662 , piece_id=PUBSU5WGV2VSPXER4D255XNPMZPBTLOLA5JS5ZBJFJWKWA6CYFBA, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.073840198 (84h51m31.796403248s) [noise=true , address=77.90.31.19:4842 , piece_id=SOQDD53C55RO67Q2C5HAYFTSKZP7ZMEWU42OON3YW24MQUMFNJ3A, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.074205587 (84h51m31.796768637s) [noise=true , address=45.91.116.11:4118 , piece_id=5OBIONSVURIRITMD2HAVHLZTIDIBQOZJENJPKAI5JVKX6NSVG4ZQ, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.142825163 (84h51m31.865388203s) [noise=true , address=45.91.116.8:4047 , piece_id=GEDNNAD5CKPNTE7I2X2IK3SCI2G2F3JPKEVXW3HLIO44XOFKIURA, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.144458218 (84h51m31.867021258s) [noise=true , address=77.90.31.16:4734 , piece_id=LDBACEVIC452LRFQ4NZWTWVJDIQZWD6HJL4XCDS55CJ362LACQBA, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.172639546 (84h51m31.895202586s) [noise=true , address=77.90.31.10:4519 , piece_id=N5HFTGGOKYCT42TILLRRHTJRYH4ZRWDQ32ADKNPGJN3ND2NQSUMQ, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.218333209 (84h51m31.940896249s) [noise=true , address=77.90.31.13:4627 , piece_id=7NR6V4GBJR5WOHQRPDO4VIWT5BPEHWJMRE36UHKIIWSI6VQE6TAA, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.223604784 (84h51m31.946167824s) [noise=true , address=77.90.31.21:4879 , piece_id=QBLWJK4D3WXQ4CO7U2XPX262ZDFPLLQKWUUUD43O5BHBCSBZBJLA, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.236036920 (84h51m31.958599961s) [noise=true , address=77.90.31.15:4698 , piece_id=X5L3W67HMYGRJVZYU72WUJ2ZT3VZ23DRBQJCNUJUV6YZCOL5ICTQ, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.251041218 (84h51m31.973604258s) [noise=true , address=45.91.116.7:4010 , piece_id=FQ35UKX2QBOJ7KNNL44QQPNJWYOPBVUKNC67BTPFVQGJ2YUM2CSA, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:22.279611289 (84h51m32.002174329s) [noise=true , address=77.90.31.11:4555 , piece_id=SZQIQFGZUB225EUHDDT4JMJYMRIJ6FE25BXHM4ZGPDJM4SGTBPQQ, seg_pos={0 0}, upload=81431]: Done uploading piece. err: <nil> Jul 28 00:27:48.947845071 (84h51m58.670408111s) [noise=true , address=104.36.82.10:4077 , piece_id=RTY63EEKUWRHDL6S5WUDY5HAGUFEGFR5BWZO2QAKCBD7UC2JCQRQ, seg_pos={0 0}, upload=81431]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: rpc: dial tcp 104.36.82.10:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:27:48.947912730 (84h51m58.67047577s) [noise=true , address=104.36.82.4:4802 , piece_id=FFNXZCMDTS632HPHFDWEOXRR3XIMFDG2UEYOAXFFESCQJIWBRWUA, seg_pos={0 0}, upload=81431]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 188.166.225.170:57482->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:27:48.947943999 (84h51m58.670507039s) [noise=true , address=104.36.82.6:4876 , piece_id=3HCF5CQHV7P73Z5I5NVFRUVGPBT5SCU24QXZGOZHRYG3MSA5HOUQ, seg_pos={0 0}, upload=81431]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 188.166.225.170:51704->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:27:48.948013092 (84h51m58.670576132s) [noise=true , address=104.36.82.2:4731 , piece_id=KO3XYSDCEOM2YFS3LJOF27UIUULA4ZVRUXFWGPJYR5RJR3BCK45Q, seg_pos={0 0}, upload=81431]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:104.36.82.2:4731): protocol: write tcp 188.166.225.170:36822->104.36.82.2:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 28 00:27:48.948060071 (84h51m58.670623121s) [seg_pos={0 0}, upload=81431]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 28 00:27:49.184974758 (84h51m58.907537808s) [upload=81431]: Done uploading segments... Jul 28 00:27:49.185004114 (84h51m58.907567154s) [upload=81431]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.007511139s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................X.......X.XX....X....] duration: 5.754663095s FAIL (2025-07-28 00:28:35 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.754689966s core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)