Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........X...............X............................X...] duration: 6.266620145s FAIL (2025-08-16 14:40:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 49dfafbb94c33c8117bc19860e46bb56246facc41c7f02daffe0cf8e29c76acb != before 1be7850c2cc74e399df94f373bb6443b6a49bf63dc3bd4b7e451763927467f44 ======LOGS====== Aug 16 14:37:37.968738630 (189h11m32.399818761s) [upload=196837]: Starting upload Aug 16 14:37:37.968781391 (189h11m32.399861513s) [upload=196837]: Done starting upload Aug 16 14:37:37.968833571 (189h11m32.399913702s) [upload=196837]: Uploading segments... Aug 16 14:37:37.968862807 (189h11m32.399942938s) [upload=196837]: Got next segment. Inline: true Aug 16 14:37:37.968866314 (189h11m32.399946445s) [upload=196837]: Waiting for error group managing segments... Aug 16 14:37:37.968908495 (189h11m32.399988646s) [upload=196837]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:37:37.968914897 (189h11m32.399995038s) [upload=196837]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:37:37.968919596 (189h11m32.399999747s) [upload=196837]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:37:41.759696736 (189h11m36.190776867s) [upload=196837]: Done uploading segments... Aug 16 14:37:41.759716192 (189h11m36.190796323s) [upload=196837]: Upload finished. err: <nil> Duration:* 6.266648458s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000683406s target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-08-16 14:39:58 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 30.000697432s edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................X.X.XX] duration: 1m0.228369794s FAIL (2025-08-16 14:39:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== 5GXTE4E2ZAUPUCMDOLZDHQAKAXZOZPHQ, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.720381011 (189h12m48.151461142s) [noise=true , address=104.36.82.17:4346 , piece_id=AAB5NDH2OSGLAE7QWUAIJSMWCAUEUTP4IARY5AMPIHL4PDY365NQ, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.720849959 (189h12m48.15193008s) [noise=true , address=79.127.220.101:4076 , piece_id=5CV4QZKRWRLK7UKHJ7RWM7DKEKL333EU6KATUBS66U2SC35AZFGQ, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.720893342 (189h12m48.151973483s) [noise=true , address=77.90.31.11:4565 , piece_id=6VQDNOSJLJUKVB7IKMYULVFVZHDUR36R4FJFAAZFFVFY6SPZVSVQ, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.721221239 (189h12m48.15230138s) [noise=true , address=77.90.31.15:4719 , piece_id=RORR374XL6TCTNYTDMDEEUJ4VFMOBUZNOGMEYFCE3NX3RPL53VBA, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.721352912 (189h12m48.152433043s) [noise=true , address=104.36.82.60:4345 , piece_id=SRDCP7N7X2I3L76HKGJQXTEK22HO2KIRMVSKKZ5EKTNWYVWWCFOQ, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.721765561 (189h12m48.152845692s) [noise=true , address=104.36.82.5:4871 , piece_id=2RKZXW25PTXKEHA32FR5D62V5VBMUXOUZ23A3TAYL2LHYXI7UTZQ, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.722196666 (189h12m48.153276797s) [noise=true , address=77.90.31.21:4899 , piece_id=6UUNM2TQGCRGMUSZU6JFK5IAAT4432RFQ4OI43E6YNTTUH5YVL2Q, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.722615467 (189h12m48.153695598s) [noise=true , address=104.36.82.59:4334 , piece_id=DANFZEPDCRN27IPYTRAVJKIANFNRPESNG7XTYADMCM7M24CMV3KA, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.722987959 (189h12m48.15406809s) [noise=true , address=77.90.31.19:4855 , piece_id=4YSZLADQ3RXAFSUPLTOHZNJ3CVIE2SXAU53OMXQJAINF7UZLXJ4A, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.723213892 (189h12m48.154294034s) [noise=true , address=77.90.31.21:4902 , piece_id=VD2RTIO6W7EAE4BLPRSX3ASHSGP5JMW5BQBJ73PLLUU633WEY5AA, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.723511853 (189h12m48.154591984s) [noise=true , address=79.127.220.98:4401 , piece_id=YLFCW6S2UWUAGMPSZFBOHXXDTNFXMQIFIAI5KXSNUDOMRP4OYHCQ, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:53.723807077 (189h12m48.154887208s) [noise=true , address=66.23.193.175:4559 , piece_id=A26T4COZ4WURGDKIVMGOI2AKXN3WFWU6QIICC5PEZAHP2W4PSYNA, seg_pos={0 0}, upload=196857]: Uploading piece... Aug 16 14:38:54.913138394 (189h12m49.344218525s) [upload=196857]: Next returned nil segment Aug 16 14:38:54.913160125 (189h12m49.344240256s) [upload=196857]: Waiting for error group managing segments... Aug 16 14:38:55.284357701 (189h12m49.715437822s) [noise=true , address=79.127.205.235:4680 , piece_id=ZDK7YJIR34MI4UYXJUDUTNVJ4IIC3VJN273T5VAPP4ONFDN3MXWA, seg_pos={0 0}, upload=196857]: Done uploading piece. err: <nil> Aug 16 14:38:55.290490025 (189h12m49.721570166s) [noise=true , address=66.23.193.175:4559 , piece_id=A26T4COZ4WURGDKIVMGOI2AKXN3WFWU6QIICC5PEZAHP2W4PSYNA, seg_pos={0 0}, upload=196857]: Done uploading piece. err: <nil> Aug 16 14:38:55.562012053 (189h12m49.993092174s) [noise=true , address=79.127.220.98:4401 , piece_id=YLFCW6S2UWUAGMPSZFBOHXXDTNFXMQIFIAI5KXSNUDOMRP4OYHCQ, seg_pos={0 0}, upload=196857]: Done uploading piece. err: <nil> Aug 16 14:38:55.613030000 (189h12m50.044110121s) [noise=true , address=79.127.220.101:4076 , piece_id=5CV4QZKRWRLK7UKHJ7RWM7DKEKL333EU6KATUBS66U2SC35AZFGQ, seg_pos={0 0}, upload=196857]: Done uploading piece. err: <nil> Aug 16 14:38:56.223897889 (189h12m50.65497802s) [noise=true , address=77.90.31.18:4840 , piece_id=JEVIKL3VCAIJ3KFE6MOFIIWTST3Q4PSSIW7GNP6ZHQRLDF7776GA, seg_pos={0 0}, upload=196857]: Done uploading piece. err: <nil> Aug 16 14:39:02.434800794 (189h12m56.865880925s) [noise=true , address=104.36.82.60:4345 , piece_id=SRDCP7N7X2I3L76HKGJQXTEK22HO2KIRMVSKKZ5EKTNWYVWWCFOQ, seg_pos={0 0}, upload=196857]: Done uploading piece. err: <nil> Aug 16 14:39:52.612997116 (189h13m47.044077247s) [noise=true , address=104.36.82.17:4346 , piece_id=AAB5NDH2OSGLAE7QWUAIJSMWCAUEUTP4IARY5AMPIHL4PDY365NQ, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza, address:104.36.82.17:4346): protocol: rpc: dial tcp 104.36.82.17:4346: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613074544 (189h13m47.044154675s) [noise=true , address=77.90.31.15:4719 , piece_id=RORR374XL6TCTNYTDMDEEUJ4VFMOBUZNOGMEYFCE3NX3RPL53VBA, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:1qyhppjd6m6he7eHVZLuhK9BYKXq1DY3GuKBNfyb97JVaz9wyk, address:77.90.31.15:4719): protocol: rpc: dial tcp 77.90.31.15:4719: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613143196 (189h13m47.044223327s) [noise=true , address=79.127.201.215:4360 , piece_id=BRUA2C2RJVDRIUQGQNNCCAN2N5JBBWJFMUJBQVV76PKVD7PSFJHA, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG, address:79.127.201.215:4360): protocol: write tcp 188.166.225.170:59584->79.127.201.215:4360: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613405698 (189h13m47.044485829s) [noise=true , address=104.36.82.5:4871 , piece_id=2RKZXW25PTXKEHA32FR5D62V5VBMUXOUZ23A3TAYL2LHYXI7UTZQ, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:1NEVGFZtdgMqREnYBPuBbYqwGRxExenRwncwCBLDFhuwdFyWdb, address:104.36.82.5:4871): protocol: rpc: dial tcp 104.36.82.5:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613521399 (189h13m47.04460153s) [noise=true , address=77.90.31.21:4902 , piece_id=VD2RTIO6W7EAE4BLPRSX3ASHSGP5JMW5BQBJ73PLLUU633WEY5AA, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:44696->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613568650 (189h13m47.044648781s) [noise=true , address=104.36.82.23:4480 , piece_id=Y54LYHUREIZ4P4QXGAPS5GXTE4E2ZAUPUCMDOLZDHQAKAXZOZPHQ, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:1qQhWAdN83i1jMTYByNQ8SWGXQaXRgBGezkLTf1BTwLyKskjjU, address:104.36.82.23:4480): protocol: rpc: dial tcp 104.36.82.23:4480: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613661226 (189h13m47.044741359s) [noise=true , address=104.36.82.59:4334 , piece_id=DANFZEPDCRN27IPYTRAVJKIANFNRPESNG7XTYADMCM7M24CMV3KA, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:12MaiBbEtrHEaXrMy5hhbYDQsGfTCZuT6C62aFxUnXPE5vRW3kg, address:104.36.82.59:4334): protocol: rpc: dial tcp 104.36.82.59:4334: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613733164 (189h13m47.044813296s) [noise=true , address=104.36.82.44:4871 , piece_id=YFVTCL2QQR6SHQALA6C62NQKREYEOPEVZDPRT43SUNCSXNAOS44A, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613786747 (189h13m47.044866878s) [noise=true , address=77.90.31.21:4899 , piece_id=6UUNM2TQGCRGMUSZU6JFK5IAAT4432RFQ4OI43E6YNTTUH5YVL2Q, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:12UjFkFn2UEvr57LDorcijDCvkBkkKitgZEFx6Yqfk5gMStQvJG, address:77.90.31.21:4899): protocol: read tcp 188.166.225.170:46556->77.90.31.21:4899: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613850388 (189h13m47.04493051s) [noise=true , address=77.90.31.19:4855 , piece_id=4YSZLADQ3RXAFSUPLTOHZNJ3CVIE2SXAU53OMXQJAINF7UZLXJ4A, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:19LE2Lq4e6HojnMp9xQ3ySLkRe5PemXTwiQZThN5jT46bQhN6K, address:77.90.31.19:4855): protocol: read tcp 188.166.225.170:54106->77.90.31.19:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613881659 (189h13m47.04496179s) [noise=true , address=77.90.31.11:4565 , piece_id=6VQDNOSJLJUKVB7IKMYULVFVZHDUR36R4FJFAAZFFVFY6SPZVSVQ, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:1xnwFcUEdmpLjmx8NXZZcpiVQwTySXDRYbchuz4oREBq1yP6wh, address:77.90.31.11:4565): protocol: rpc: dial tcp 77.90.31.11:4565: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613965309 (189h13m47.04504543s) [noise=true , address=104.36.82.34:4554 , piece_id=PF4YCVD5XOD4OZ5GZHXXZYGKIOBQ3MRC72QA7FDK54S6BUW7SSMQ, seg_pos={0 0}, upload=196857]: Done uploading piece. err: ecclient: upload failed (node:12rSDkPXm47w73hry9W6BWwAiwNpB3cJuwedNt1JuSEEtd3Sup7, address:104.36.82.34:4554): protocol: read tcp 188.166.225.170:41242->104.36.82.34:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:52.613991680 (189h13m47.045071811s) [seg_pos={0 0}, upload=196857]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 16 14:39:52.840167622 (189h13m47.271247743s) [upload=196857]: Done uploading segments... Aug 16 14:39:52.840195957 (189h13m47.271276088s) [upload=196857]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.228383461s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...........................................X........XXX..XXX] duration: 30.000293861s FAIL (2025-08-16 14:39:41 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 30.000312125s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000477318s FAIL (2025-08-16 14:39:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 30.000505934s edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X...............X............................X..] duration: 1.82436788s FAIL (2025-08-16 14:39:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 49dfafbb94c33c8117bc19860e46bb56246facc41c7f02daffe0cf8e29c76acb != before 1be7850c2cc74e399df94f373bb6443b6a49bf63dc3bd4b7e451763927467f44 ======LOGS====== Aug 16 14:37:37.968738630 (189h11m32.399818761s) [upload=196837]: Starting upload Aug 16 14:37:37.968781391 (189h11m32.399861513s) [upload=196837]: Done starting upload Aug 16 14:37:37.968833571 (189h11m32.399913702s) [upload=196837]: Uploading segments... Aug 16 14:37:37.968862807 (189h11m32.399942938s) [upload=196837]: Got next segment. Inline: true Aug 16 14:37:37.968866314 (189h11m32.399946445s) [upload=196837]: Waiting for error group managing segments... Aug 16 14:37:37.968908495 (189h11m32.399988646s) [upload=196837]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:37:37.968914897 (189h11m32.399995038s) [upload=196837]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:37:37.968919596 (189h11m32.399999747s) [upload=196837]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:37:41.759696736 (189h11m36.190776867s) [upload=196837]: Done uploading segments... Aug 16 14:37:41.759716192 (189h11m36.190796323s) [upload=196837]: Upload finished. err: <nil> Duration:* 1.82438413s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................................................XXX..XX] duration: 30.225873027s FAIL (2025-08-16 14:39:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 2) ======LOGS====== 58.837004627 (189h12m53.268084748s) [noise=true , address=104.36.82.11:4115 , piece_id=L6BBN3LXH7JQSMLAUUBLJZTWV5TEEG3B7VKPMGNVUC2RG5MKDEBQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.837256408 (189h12m53.268336529s) [noise=true , address=104.36.82.10:4079 , piece_id=S5OW3P54VHUHZTYAPUXEKCPR3G5Q6ASAJ7BFVLGQ65UE5VEDT3RQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.837477842 (189h12m53.268557973s) [noise=true , address=104.36.82.13:4187 , piece_id=HJ7YZRO422DXRHAPEV3RQ3LU5O6CIWY5RCNHWKKEOXC4YAPXUMLA, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.837736777 (189h12m53.268816908s) [noise=true , address=77.90.31.10:4519 , piece_id=GIIAIWO2CLAZ6OWUMJVKFFLQTN4GGB3GYRTZJKEP5KPURSXPLSEQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.837886464 (189h12m53.268966595s) [noise=true , address=104.36.82.8:4971 , piece_id=Y3JBTLY6AIYSJWTFU4EBWXVCO27YWT555YCWVJDBXDXPV6H7IT6A, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.838420225 (189h12m53.269500356s) [noise=true , address=104.36.82.15:4258 , piece_id=7UV55IXSO4FGL2OP5GWQNLVSWAUC7PR2J2TYGXBC6MQ37TLKYOOQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.838433771 (189h12m53.269513902s) [noise=true , address=104.36.82.14:4222 , piece_id=5VYI6YLESBTEOHWDBQ5Q67YSXXY4VGXHXBN4HO7WSFKUG3GTK7RA, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.839011426 (189h12m53.270091557s) [noise=true , address=104.36.82.6:4873 , piece_id=GM6QQE7M2ZXSMYSMQHS4S77W6PWEMIB3WIRNQ7SAYWP74HEEYCDQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.839708480 (189h12m53.270788611s) [noise=true , address=45.91.116.10:4082 , piece_id=7XELIZWK4XW5E6O5HVWCPLFPIRGHF4R47ZU56S5SOVV5KHPZLEZQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.840424069 (189h12m53.2715042s) [noise=true , address=104.36.82.2:4729 , piece_id=LRL25FR42O7G4UNBVEMV5MKRACHBPCVKOSY2PIKCDMOXI62Q5IXQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.841035189 (189h12m53.27211531s) [noise=true , address=77.90.31.15:4697 , piece_id=YBVNE4POO22WD24BB7U5KBOGEP37HH6N5HVVZU6AOE5EUJNVKGJQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.841360742 (189h12m53.272440873s) [noise=true , address=104.36.82.10:4077 , piece_id=JUWKSRALHSRTEDRZP3S3Y7MH4DKJKPTFGR7GCUHYXLT5QBFZXJDQ, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:38:58.841851351 (189h12m53.272931472s) [noise=true , address=104.36.82.12:4149 , piece_id=D4CUFEPTZIG3DLPSGE3BOM6DTAE2OCBDQ5NAMC7IIMDV7QWWAG5Q, seg_pos={0 0}, upload=196859]: Uploading piece... Aug 16 14:39:01.494200350 (189h12m55.925280481s) [upload=196859]: Next returned nil segment Aug 16 14:39:01.494217032 (189h12m55.925297163s) [upload=196859]: Waiting for error group managing segments... Aug 16 14:39:01.882160970 (189h12m56.313241111s) [noise=true , address=104.36.82.4:4803 , piece_id=4YCJQDUTOPCCOVLSUONV4HULSULVFR5KRPZUQ26EVVEXMV63PYEQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: <nil> Aug 16 14:39:25.459729150 (189h13m19.890809281s) [noise=true , address=104.36.82.7:4912 , piece_id=BU67HU3CFJNT654XTV6MGF4R6GVM5UMKRLVE5Y3CHULRRSFLEJWQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: <nil> Aug 16 14:39:27.961074051 (189h13m22.392154172s) [noise=true , address=104.36.82.12:4149 , piece_id=D4CUFEPTZIG3DLPSGE3BOM6DTAE2OCBDQ5NAMC7IIMDV7QWWAG5Q, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: read tcp 188.166.225.170:59060->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961178280 (189h13m22.392258401s) [noise=true , address=45.91.116.10:4082 , piece_id=7XELIZWK4XW5E6O5HVWCPLFPIRGHF4R47ZU56S5SOVV5KHPZLEZQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961216393 (189h13m22.392296524s) [noise=true , address=104.36.82.2:4729 , piece_id=LRL25FR42O7G4UNBVEMV5MKRACHBPCVKOSY2PIKCDMOXI62Q5IXQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: rpc: dial tcp 104.36.82.2:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961313459 (189h13m22.39239359s) [noise=true , address=104.36.82.10:4077 , piece_id=JUWKSRALHSRTEDRZP3S3Y7MH4DKJKPTFGR7GCUHYXLT5QBFZXJDQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: rpc: dial tcp 104.36.82.10:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961322767 (189h13m22.392402908s) [noise=true , address=104.36.82.11:4115 , piece_id=L6BBN3LXH7JQSMLAUUBLJZTWV5TEEG3B7VKPMGNVUC2RG5MKDEBQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961368384 (189h13m22.392448515s) [noise=true , address=104.36.82.13:4187 , piece_id=HJ7YZRO422DXRHAPEV3RQ3LU5O6CIWY5RCNHWKKEOXC4YAPXUMLA, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961377321 (189h13m22.392457452s) [noise=true , address=104.36.82.10:4079 , piece_id=S5OW3P54VHUHZTYAPUXEKCPR3G5Q6ASAJ7BFVLGQ65UE5VEDT3RQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961409913 (189h13m22.392490044s) [noise=true , address=104.36.82.8:4971 , piece_id=Y3JBTLY6AIYSJWTFU4EBWXVCO27YWT555YCWVJDBXDXPV6H7IT6A, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: rpc: dial tcp 104.36.82.8:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961419762 (189h13m22.392499883s) [noise=true , address=104.36.82.6:4873 , piece_id=GM6QQE7M2ZXSMYSMQHS4S77W6PWEMIB3WIRNQ7SAYWP74HEEYCDQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: rpc: dial tcp 104.36.82.6:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961467503 (189h13m22.392547634s) [noise=true , address=104.36.82.14:4222 , piece_id=5VYI6YLESBTEOHWDBQ5Q67YSXXY4VGXHXBN4HO7WSFKUG3GTK7RA, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961476731 (189h13m22.392556863s) [noise=true , address=77.90.31.10:4519 , piece_id=GIIAIWO2CLAZ6OWUMJVKFFLQTN4GGB3GYRTZJKEP5KPURSXPLSEQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961508642 (189h13m22.392588763s) [noise=true , address=77.90.31.15:4697 , piece_id=YBVNE4POO22WD24BB7U5KBOGEP37HH6N5HVVZU6AOE5EUJNVKGJQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: rpc: dial tcp 77.90.31.15:4697: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961526115 (189h13m22.392606236s) [noise=true , address=104.36.82.15:4258 , piece_id=7UV55IXSO4FGL2OP5GWQNLVSWAUC7PR2J2TYGXBC6MQ37TLKYOOQ, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961572204 (189h13m22.392652325s) [noise=true , address=45.91.116.14:4189 , piece_id=MGIEKL3AX6IZBA5MGYCK5CP3ARRTE7BO4OO36NGDCJSKXF2GRRTA, seg_pos={0 0}, upload=196859]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:49734->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:27.961591481 (189h13m22.392671612s) [seg_pos={0 0}, upload=196859]: Done waiting for segment. successful: 2 optimal: 13 errs: <nil> Aug 16 14:39:28.184232199 (189h13m22.61531233s) [upload=196859]: Done uploading segments... Aug 16 14:39:28.184253119 (189h13m22.61533325s) [upload=196859]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 2) Duration: 30.225886131s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [........................................................X.XX] duration: 30.000367079s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-16 14:39:17 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 30.000383731s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XX..X] duration: 30.000975176s FAIL (2025-08-16 14:39:13 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 30.001006235s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................................X........XXX..XX] duration: 30.001616987s FAIL (2025-08-16 14:39:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001629742s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X...X...........X................................X.X.X] duration: 1m0.449924511s FAIL (2025-08-16 14:39:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== Aug 16 14:38:08.151003134 (189h12m2.582083265s) [upload=196845]: Starting upload Aug 16 14:38:08.151054983 (189h12m2.582135114s) [upload=196845]: Done starting upload Aug 16 14:38:08.151241059 (189h12m2.58232123s) [upload=196845]: Uploading segments... Aug 16 14:38:08.151818785 (189h12m2.582898906s) [upload=196845]: Got next segment. Inline: false Aug 16 14:38:08.151877266 (189h12m2.582957397s) [upload=196845]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:38:08.151882046 (189h12m2.582962177s) [upload=196845]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 14:38:08.901910885 (189h12m3.332991006s) [seg_pos={0 0}, upload=196845]: Begin upload segment... Aug 16 14:38:08.901952885 (189h12m3.333033016s) [seg_pos={0 0}, upload=196845]: Done begin upload segment. Aug 16 14:38:08.901974676 (189h12m3.333054808s) [noise=true , address=104.36.82.9:4008 , piece_id=KMZGEG63XWFPHMGRYPDAKYLYLVFMSNPNFZUF7N24SLQK4WN3WBYA, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.901993072 (189h12m3.333073203s) [noise=true , address=104.36.82.60:4349 , piece_id=5D5MEJ2UYVYNWR4LG2F3OVGTEMZ4UAM533S7PDVL4LSAIQB3ACAA, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.902331770 (189h12m3.333411901s) [noise=true , address=45.91.116.5:4329 , piece_id=Z6WWFRDGJG5R3AKLXVKQ5PEAERTPVH5AM7ZDIXOCGPHZJZRYOOTQ, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.902343513 (189h12m3.333423644s) [noise=true , address=104.36.82.5:4862 , piece_id=X3FNG7RMX2W4H4XOVBQPJYZ4W4WN4JKPYWWCMK76RNBGKDJRLEKQ, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.902676259 (189h12m3.33375638s) [noise=true , address=66.23.193.175:4557 , piece_id=SERURIXEIMXUX6CIXCPIKTQPNYA632YI25YAJ4GMNP2UHYRBHSPQ, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.902693752 (189h12m3.333773883s) [noise=true , address=45.91.116.12:4231 , piece_id=IB467SQ5Y7EVPF5LVUD2UIYYJPDAQ4VBQGPWUGKMXXMOG7EGRUOA, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.902983557 (189h12m3.334063688s) [noise=true , address=104.36.82.59:4315 , piece_id=O7PBBCEOZF7NBFH5NXYQLRVWEUKGB6DASOS3B7FFFOELLMNNUUYA, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.903007944 (189h12m3.334088065s) [noise=true , address=104.36.82.4:4808 , piece_id=KZVSTTAD5J35NNT4XKZLOYQ7LWA3IVKJ6LLAMS5H5JHTIFDSQ62Q, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.903278522 (189h12m3.334358653s) [noise=true , address=77.90.31.16:4745 , piece_id=4Z2A4WPT6QIJJ2EDCKJMO2RFJ7CN2ZEFCWTHZG2YV46K33N4HEGA, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.903294963 (189h12m3.334375094s) [noise=true , address=104.36.82.35:4612 , piece_id=HAP6IT3LYPF27PLUA3CGL2FJSDXSNIIN7YU746MUFKX3GKUSLBNQ, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.903537948 (189h12m3.334618079s) [noise=true , address=77.90.31.21:4907 , piece_id=NHFTTJ5AQUPM2XGD66GLMYBVXJ7NM4AIKFNCZO2YHDJAY2Q6BFSQ, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.903572333 (189h12m3.334652464s) [noise=true , address=104.36.82.44:4901 , piece_id=ZRHCK3ZZ4KXNXXDBDY5P3HSHN52CNQFL2BS2DRDTXKLHAGM3SR4Q, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.903744513 (189h12m3.334824634s) [noise=true , address=79.127.201.215:4354 , piece_id=YHK7PNPMNF65SIP6GMKCNGPAZMCA5X4BAJSX44M3DFVRHCP3CQAQ, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.903800490 (189h12m3.334880621s) [noise=true , address=77.90.31.15:4726 , piece_id=WOIGX55IVDXCO4E3QPDETOAYHE6QSWLQPPLJNOQ74IGDO24DCYDA, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.903948814 (189h12m3.335028945s) [noise=true , address=104.36.82.23:4462 , piece_id=OOGM4OTCRU7H4SXOBULAGDYFZJ5XC74NK7SLPDIUB4ZT6ZOALGTA, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.904035379 (189h12m3.33511551s) [noise=true , address=79.127.220.98:4401 , piece_id=OC2BYK3RFYNQHSJHIW4KUUJEUHZSX4VSRHVHU3CJ27VKQR3ZKAYQ, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.904157163 (189h12m3.335237284s) [noise=true , address=104.36.82.38:4706 , piece_id=ZDHGGG526SRLY4DJ43HQOXXEI5NMJUJXRYDZOV5FTIJ3T5T4B7LQ, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:08.904389427 (189h12m3.335469558s) [noise=true , address=79.127.220.101:4071 , piece_id=QY4FTEIVHWAS6SLFJTUZZ4AHX7WLLGMCMLPE6GIFRWX2FQV725BA, seg_pos={0 0}, upload=196845]: Uploading piece... Aug 16 14:38:10.064224826 (189h12m4.495304957s) [upload=196845]: Next returned nil segment Aug 16 14:38:10.064600194 (189h12m4.495680315s) [upload=196845]: Waiting for error group managing segments... Aug 16 14:38:10.359465336 (189h12m4.790545457s) [noise=true , address=66.23.193.175:4557 , piece_id=SERURIXEIMXUX6CIXCPIKTQPNYA632YI25YAJ4GMNP2UHYRBHSPQ, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:10.748440177 (189h12m5.179520298s) [noise=true , address=79.127.220.101:4071 , piece_id=QY4FTEIVHWAS6SLFJTUZZ4AHX7WLLGMCMLPE6GIFRWX2FQV725BA, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:10.827568591 (189h12m5.258648732s) [noise=true , address=79.127.220.98:4401 , piece_id=OC2BYK3RFYNQHSJHIW4KUUJEUHZSX4VSRHVHU3CJ27VKQR3ZKAYQ, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:10.881388199 (189h12m5.31246833s) [noise=true , address=79.127.201.215:4354 , piece_id=YHK7PNPMNF65SIP6GMKCNGPAZMCA5X4BAJSX44M3DFVRHCP3CQAQ, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:11.130665564 (189h12m5.561745695s) [noise=true , address=104.36.82.59:4315 , piece_id=O7PBBCEOZF7NBFH5NXYQLRVWEUKGB6DASOS3B7FFFOELLMNNUUYA, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:11.186702865 (189h12m5.617782996s) [noise=true , address=104.36.82.38:4706 , piece_id=ZDHGGG526SRLY4DJ43HQOXXEI5NMJUJXRYDZOV5FTIJ3T5T4B7LQ, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:14.859654005 (189h12m9.290734136s) [noise=true , address=104.36.82.4:4808 , piece_id=KZVSTTAD5J35NNT4XKZLOYQ7LWA3IVKJ6LLAMS5H5JHTIFDSQ62Q, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:15.220659810 (189h12m9.651739952s) [noise=true , address=104.36.82.44:4901 , piece_id=ZRHCK3ZZ4KXNXXDBDY5P3HSHN52CNQFL2BS2DRDTXKLHAGM3SR4Q, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:15.683697862 (189h12m10.114777993s) [noise=true , address=45.91.116.5:4329 , piece_id=Z6WWFRDGJG5R3AKLXVKQ5PEAERTPVH5AM7ZDIXOCGPHZJZRYOOTQ, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:22.464847704 (189h12m16.895927825s) [noise=true , address=104.36.82.9:4008 , piece_id=KMZGEG63XWFPHMGRYPDAKYLYLVFMSNPNFZUF7N24SLQK4WN3WBYA, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:27.950218202 (189h12m22.381298333s) [noise=true , address=104.36.82.35:4612 , piece_id=HAP6IT3LYPF27PLUA3CGL2FJSDXSNIIN7YU746MUFKX3GKUSLBNQ, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:38:36.637099148 (189h12m31.068179279s) [noise=true , address=77.90.31.21:4907 , piece_id=NHFTTJ5AQUPM2XGD66GLMYBVXJ7NM4AIKFNCZO2YHDJAY2Q6BFSQ, seg_pos={0 0}, upload=196845]: Done uploading piece. err: <nil> Aug 16 14:39:07.492440376 (189h13m1.923520497s) [noise=true , address=77.90.31.16:4745 , piece_id=4Z2A4WPT6QIJJ2EDCKJMO2RFJ7CN2ZEFCWTHZG2YV46K33N4HEGA, seg_pos={0 0}, upload=196845]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: write tcp 188.166.225.170:49774->77.90.31.16:4745: use of closed network connection; write tcp 188.166.225.170:49774->77.90.31.16:4745: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49774->77.90.31.16:4745: use of closed network connection Aug 16 14:39:07.492635248 (189h13m1.923715369s) [noise=true , address=104.36.82.60:4349 , piece_id=5D5MEJ2UYVYNWR4LG2F3OVGTEMZ4UAM533S7PDVL4LSAIQB3ACAA, seg_pos={0 0}, upload=196845]: Done uploading piece. err: ecclient: upload failed (node:12CrzYmDPm95TgyE7SHJsFVbL1JUTdwMmZVge4uEkSCbeP8iBEk, address:104.36.82.60:4349): protocol: write tcp 188.166.225.170:42588->104.36.82.60:4349: use of closed network connection; write tcp 188.166.225.170:42588->104.36.82.60:4349: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42588->104.36.82.60:4349: use of closed network connection Aug 16 14:39:07.492728317 (189h13m1.923808448s) [noise=true , address=104.36.82.23:4462 , piece_id=OOGM4OTCRU7H4SXOBULAGDYFZJ5XC74NK7SLPDIUB4ZT6ZOALGTA, seg_pos={0 0}, upload=196845]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: write tcp 188.166.225.170:55050->104.36.82.23:4462: use of closed network connection; write tcp 188.166.225.170:55050->104.36.82.23:4462: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55050->104.36.82.23:4462: use of closed network connection Aug 16 14:39:07.492765348 (189h13m1.923845479s) [noise=true , address=77.90.31.15:4726 , piece_id=WOIGX55IVDXCO4E3QPDETOAYHE6QSWLQPPLJNOQ74IGDO24DCYDA, seg_pos={0 0}, upload=196845]: Done uploading piece. err: ecclient: upload failed (node:12kMaukCkCJSc3ukAZu3thLgzXGGzNq44dGyEjCPqoY3PSYVEm3, address:77.90.31.15:4726): protocol: read tcp 188.166.225.170:53996->77.90.31.15:4726: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:07.492798821 (189h13m1.923878952s) [noise=true , address=45.91.116.12:4231 , piece_id=IB467SQ5Y7EVPF5LVUD2UIYYJPDAQ4VBQGPWUGKMXXMOG7EGRUOA, seg_pos={0 0}, upload=196845]: Done uploading piece. err: ecclient: upload failed (node:1NmgZdLWKjrDMmNcgmnzH475akX4WtA9zGTLxZgVW7JpaecBjq, address:45.91.116.12:4231): protocol: read tcp 188.166.225.170:36228->45.91.116.12:4231: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:07.492831795 (189h13m1.923911926s) [noise=true , address=104.36.82.5:4862 , piece_id=X3FNG7RMX2W4H4XOVBQPJYZ4W4WN4JKPYWWCMK76RNBGKDJRLEKQ, seg_pos={0 0}, upload=196845]: Done uploading piece. err: ecclient: upload failed (node:12sUT7LBYFqWMUhx7MMjEmyiFDnbJnjSVndNbTaHioaVLLSXYue, address:104.36.82.5:4862): protocol: read tcp 188.166.225.170:53704->104.36.82.5:4862: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:39:07.492845260 (189h13m1.923925382s) [seg_pos={0 0}, upload=196845]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 16 14:39:07.939954375 (189h13m2.371034506s) [upload=196845]: Done uploading segments... Aug 16 14:39:07.939979994 (189h13m2.371060125s) [upload=196845]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.449948176s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XXX..X] duration: 30.230871159s FAIL (2025-08-16 14:38:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 61 , piece_id=TEV7WFGPMNWXZ5U3OJQO5UFSEQ2KLXKZSMT3M4FJCBYYSYISU6FQ, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.607304946 (189h12m23.038385077s) [noise=true , address=104.36.82.11:4114 , piece_id=4MDYK7SFDVNOP64CQ4TT5JZCGVMBWFPIJ3RF723GEE6RSMBAK2YA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.607373857 (189h12m23.038453988s) [noise=true , address=45.91.116.11:4118 , piece_id=TIVE47XVPMFDP3ALBEJ2ICZ772BJ56ALBAY7QYPK7BYKRBYGBJQA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.607639055 (189h12m23.038719176s) [noise=true , address=104.36.82.10:4079 , piece_id=PZZ2GRNWIE54I4VZ64GSZY4ALU67KWRMARPXIEQJDOFBTQ7BOHCA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.607829690 (189h12m23.038909821s) [noise=true , address=77.90.31.15:4698 , piece_id=OLQWRAQXT3NMBEOLOOTBYLBORHMFTQCJXD5EM6VKM27R3XZ422SA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.607912889 (189h12m23.03899301s) [noise=true , address=45.91.116.14:4190 , piece_id=J3W4F4DEDJRMO2N6HHIV6E7QWOMKTKZCT2XGR273CB7BXMZVZJJA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.608170201 (189h12m23.039250332s) [noise=true , address=104.36.82.7:4912 , piece_id=ILEBNMPU7AQWGJ2WFLEM5ZJ3XLMS2ASEL7PR6EVBTY573LX4QJXA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.608420611 (189h12m23.039500782s) [noise=true , address=104.36.82.12:4151 , piece_id=D4NQQ5DXBGW4MZM2IQJ2D5HBO4MCHDPMRSISZ5QZH5M6CANQOI5A, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.608477329 (189h12m23.03955745s) [noise=true , address=104.36.82.15:4257 , piece_id=LIPHLPFATOFWJZF4J76AINAUIS5VSHIIVDSX6KPLRI6OVCHHCYDA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.608686008 (189h12m23.039766139s) [noise=true , address=104.36.82.5:4839 , piece_id=HRK5K7DHNEAHKWT5K253NZGXY5WFJGUQC5FBIC5WA7HRYFUPJVJQ, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.608763657 (189h12m23.039843788s) [noise=true , address=104.36.82.16:4294 , piece_id=STGMNSHI25GDZXT3ERUVDAXDYA663DUSGDWJRS3YYZ2IFOR6FBZA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.608887634 (189h12m23.039967755s) [noise=true , address=104.36.82.14:4222 , piece_id=Y4AW3OASTCMTWWJNIRWHOE7KY5V6NU3VVC2Z3SZO4J42AECIEMIA, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.609100952 (189h12m23.040181083s) [noise=true , address=104.36.82.8:4972 , piece_id=B64SC5HA2QAD56PVIJIYWP5MOMRHQKQZOFQH64K64WR5ZPELY24A, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:28.609302728 (189h12m23.040382859s) [noise=true , address=104.36.82.8:4969 , piece_id=AE7G2AEJOXLIJW6HJNECX4QHO5HSXZZCEK4DUJWKEC2LZNUKV64Q, seg_pos={0 0}, upload=196851]: Uploading piece... Aug 16 14:38:30.469886435 (189h12m24.900966566s) [upload=196851]: Next returned nil segment Aug 16 14:38:30.469903978 (189h12m24.900984109s) [upload=196851]: Waiting for error group managing segments... Aug 16 14:38:30.768921506 (189h12m25.200001637s) [noise=true , address=104.36.82.15:4257 , piece_id=LIPHLPFATOFWJZF4J76AINAUIS5VSHIIVDSX6KPLRI6OVCHHCYDA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: <nil> Aug 16 14:38:32.325150942 (189h12m26.756231073s) [noise=true , address=77.90.31.15:4698 , piece_id=OLQWRAQXT3NMBEOLOOTBYLBORHMFTQCJXD5EM6VKM27R3XZ422SA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: <nil> Aug 16 14:38:32.847922188 (189h12m27.279002309s) [noise=true , address=45.91.116.9:4261 , piece_id=TEV7WFGPMNWXZ5U3OJQO5UFSEQ2KLXKZSMT3M4FJCBYYSYISU6FQ, seg_pos={0 0}, upload=196851]: Done uploading piece. err: <nil> Aug 16 14:38:40.834000912 (189h12m35.265081043s) [noise=true , address=104.36.82.11:4114 , piece_id=4MDYK7SFDVNOP64CQ4TT5JZCGVMBWFPIJ3RF723GEE6RSMBAK2YA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: <nil> Aug 16 14:38:43.340894372 (189h12m37.771974493s) [noise=true , address=104.36.82.14:4222 , piece_id=Y4AW3OASTCMTWWJNIRWHOE7KY5V6NU3VVC2Z3SZO4J42AECIEMIA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: <nil> Aug 16 14:38:57.502769641 (189h12m51.933849772s) [noise=true , address=104.36.82.8:4972 , piece_id=B64SC5HA2QAD56PVIJIYWP5MOMRHQKQZOFQH64K64WR5ZPELY24A, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: read tcp 188.166.225.170:33492->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:57.503458340 (189h12m51.934538471s) [noise=true , address=104.36.82.8:4969 , piece_id=AE7G2AEJOXLIJW6HJNECX4QHO5HSXZZCEK4DUJWKEC2LZNUKV64Q, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: read tcp 188.166.225.170:50964->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:57.503494539 (189h12m51.93457467s) [noise=true , address=104.36.82.13:4187 , piece_id=WXQA75RU3R44PT5YK43A2KUKCMHGJIQVAX7VWHMUADHGR3O3L4XQ, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: read tcp 188.166.225.170:34918->104.36.82.13:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:57.503523775 (189h12m51.934603906s) [noise=true , address=104.36.82.10:4079 , piece_id=PZZ2GRNWIE54I4VZ64GSZY4ALU67KWRMARPXIEQJDOFBTQ7BOHCA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: read tcp 188.166.225.170:42862->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:57.503553251 (189h12m51.934633382s) [noise=true , address=45.91.116.14:4190 , piece_id=J3W4F4DEDJRMO2N6HHIV6E7QWOMKTKZCT2XGR273CB7BXMZVZJJA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 188.166.225.170:60582->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:57.503582858 (189h12m51.934662979s) [noise=true , address=104.36.82.5:4839 , piece_id=HRK5K7DHNEAHKWT5K253NZGXY5WFJGUQC5FBIC5WA7HRYFUPJVJQ, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: read tcp 188.166.225.170:55072->104.36.82.5:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:57.503611853 (189h12m51.934691984s) [noise=true , address=104.36.82.7:4912 , piece_id=ILEBNMPU7AQWGJ2WFLEM5ZJ3XLMS2ASEL7PR6EVBTY573LX4QJXA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: read tcp 188.166.225.170:42580->104.36.82.7:4912: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:57.503641780 (189h12m51.934721911s) [noise=true , address=104.36.82.16:4294 , piece_id=STGMNSHI25GDZXT3ERUVDAXDYA663DUSGDWJRS3YYZ2IFOR6FBZA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: read tcp 188.166.225.170:51914->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:57.503720110 (189h12m51.934800241s) [noise=true , address=45.91.116.11:4118 , piece_id=TIVE47XVPMFDP3ALBEJ2ICZ772BJ56ALBAY7QYPK7BYKRBYGBJQA, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 188.166.225.170:60026->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:60026->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60026->45.91.116.11:4118: use of closed network connection Aug 16 14:38:57.503784724 (189h12m51.934864855s) [noise=true , address=104.36.82.12:4151 , piece_id=D4NQQ5DXBGW4MZM2IQJ2D5HBO4MCHDPMRSISZ5QZH5M6CANQOI5A, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:43720->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:43720->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43720->104.36.82.12:4151: use of closed network connection Aug 16 14:38:57.503899344 (189h12m51.934979475s) [noise=true , address=104.36.82.12:4150 , piece_id=J76RKL4JTHD4QLPYZYLJVYCK5PXJEZCTU2EDIXHVUTYHI54LIJXQ, seg_pos={0 0}, upload=196851]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: context deadline exceeded; context deadline exceeded Aug 16 14:38:57.503920814 (189h12m51.935000935s) [seg_pos={0 0}, upload=196851]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Aug 16 14:38:57.732081575 (189h12m52.163161706s) [upload=196851]: Done uploading segments... Aug 16 14:38:57.732106292 (189h12m52.163186424s) [upload=196851]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.230887942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................X.X.X] duration: 1m0.231828146s FAIL (2025-08-16 14:38:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== 596928 (189h11m47.683677059s) [noise=true , address=104.36.82.4:4809 , piece_id=NNXWZXAQZRY2XQ4XSPYNFQMKN2BP7LIJSRJDPFIMMZWR5QU5Y7EQ, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.252702400 (189h11m47.683782521s) [noise=true , address=104.36.82.58:4298 , piece_id=DZIAHLCPRWCHJY6YLXJ27R52V5QH232NK7Q3ZK27VJOVCASOOOZQ, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.252893385 (189h11m47.683973506s) [noise=true , address=104.36.82.39:4754 , piece_id=PHK3X4GRDU66OINBECGGEVP2S6UER5W4HODYMHWKC3A2JPO3YPRA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.252963189 (189h11m47.68404332s) [noise=true , address=104.36.82.23:4476 , piece_id=5T42PWSGTXMKXI5HNT47K2ZAWA6K47JV3FI23Z7TWLOY7MYQPRCA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253159545 (189h11m47.684239676s) [noise=true , address=45.91.116.10:4107 , piece_id=PQE22NFIZUFM4MSWTNF4KKNTYXOXPY2H6Y5CSBL4G3GZVJCVBZYA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253235110 (189h11m47.684315231s) [noise=true , address=104.36.82.5:4863 , piece_id=GUKXQW4WKA3NBYFWBN5VU5USJ5GJHGTVCMBUMISVFJZEWJ33OR5A, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253427598 (189h11m47.684507719s) [noise=true , address=45.91.116.5:4300 , piece_id=UXOWJ3SOJKKW56O4BE3JQPYAPFNNELQLQLKJG6DXECAS5UAUXSHA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253493484 (189h11m47.684573615s) [noise=true , address=77.90.31.19:4845 , piece_id=VOVPPIYLWCVKXRRH6SILENB5ZPYK7PL274CAOOS2S5YGO2ZK3BCA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253662697 (189h11m47.684742828s) [noise=true , address=64.49.13.27:4957 , piece_id=GAR2RVVKA65EICURRKEHASEL7R5GF57JQ34O7AG7BFEBZRREDNRA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253696582 (189h11m47.684776703s) [noise=true , address=104.36.82.44:4889 , piece_id=P4SO6FM4YAKS4ETAICGUV2UQSP2I5ZQH7XDEO6INOXNTFS3VKSCA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253927855 (189h11m47.685007986s) [noise=true , address=79.127.201.215:4355 , piece_id=JQZ7LAECBG4AG7BR7EIIX4XPK6GGW7EZ6O2YLAORZIPQUKJ7IZGQ, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253931301 (189h11m47.685011433s) [noise=true , address=104.36.82.60:4344 , piece_id=UXFEVKB7PMIYGFOP43KHQSB7FWTUBCMX2JOPKZOYCSKXRW2PGAKA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.253963553 (189h11m47.685043674s) [noise=true , address=77.90.31.14:4689 , piece_id=74GUX7DCA2NY3O2B7C2WBBVAK4Q7BSPC3Q7UGUP63HHC5JU4U24A, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.254177262 (189h11m47.685257393s) [noise=true , address=45.91.116.7:4016 , piece_id=OXIRHX36DTUQ3QGNMP4IRZPXL3XEI3D6XYRIABNVKLC2ECMIH3PQ, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:53.254420648 (189h11m47.685500769s) [noise=true , address=77.90.31.16:4756 , piece_id=KTJ5YSYMK3UK7UEOFFGAJI4DQ3PUI5YDJCZPATTEUG2TNVDG5OSA, seg_pos={0 0}, upload=196840]: Uploading piece... Aug 16 14:37:54.367352702 (189h11m48.798432833s) [upload=196840]: Next returned nil segment Aug 16 14:37:54.367378431 (189h11m48.798458612s) [upload=196840]: Waiting for error group managing segments... Aug 16 14:37:54.673019622 (189h11m49.104099753s) [noise=true , address=64.49.13.27:4957 , piece_id=GAR2RVVKA65EICURRKEHASEL7R5GF57JQ34O7AG7BFEBZRREDNRA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:37:54.682636463 (189h11m49.113716594s) [noise=true , address=66.23.193.175:4562 , piece_id=6QDXYAXM7O4MSFZXLBFA36M774DSEQFS663YTCG7UBD22DN42BBA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:37:54.692050687 (189h11m49.123130818s) [noise=true , address=79.127.205.235:4680 , piece_id=I5OJ6U45RV43DKGKZF2P7AIRKJZJWKCQVRJCEIGKVSF4XMUXLFRA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:37:55.381936968 (189h11m49.813017099s) [noise=true , address=79.127.201.215:4355 , piece_id=JQZ7LAECBG4AG7BR7EIIX4XPK6GGW7EZ6O2YLAORZIPQUKJ7IZGQ, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:37:57.017107398 (189h11m51.448187529s) [noise=true , address=104.36.82.5:4863 , piece_id=GUKXQW4WKA3NBYFWBN5VU5USJ5GJHGTVCMBUMISVFJZEWJ33OR5A, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:38:01.645860903 (189h11m56.076941034s) [noise=true , address=45.91.116.5:4300 , piece_id=UXOWJ3SOJKKW56O4BE3JQPYAPFNNELQLQLKJG6DXECAS5UAUXSHA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:38:02.068115531 (189h11m56.499195662s) [noise=true , address=77.90.31.16:4756 , piece_id=KTJ5YSYMK3UK7UEOFFGAJI4DQ3PUI5YDJCZPATTEUG2TNVDG5OSA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:38:03.414398135 (189h11m57.845478266s) [noise=true , address=45.91.116.10:4107 , piece_id=PQE22NFIZUFM4MSWTNF4KKNTYXOXPY2H6Y5CSBL4G3GZVJCVBZYA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:38:07.705200098 (189h12m2.136280219s) [noise=true , address=104.36.82.44:4889 , piece_id=P4SO6FM4YAKS4ETAICGUV2UQSP2I5ZQH7XDEO6INOXNTFS3VKSCA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: <nil> Aug 16 14:38:52.152866513 (189h12m46.583946644s) [noise=true , address=104.36.82.39:4754 , piece_id=PHK3X4GRDU66OINBECGGEVP2S6UER5W4HODYMHWKC3A2JPO3YPRA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:1HB1v3ZURPMSYsbHBjJs1neE7HQtLS1KmeLwbKK2sTKAZcXPhi, address:104.36.82.39:4754): protocol: read tcp 188.166.225.170:53794->104.36.82.39:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.152930194 (189h12m46.584010325s) [noise=true , address=45.91.116.7:4016 , piece_id=OXIRHX36DTUQ3QGNMP4IRZPXL3XEI3D6XYRIABNVKLC2ECMIH3PQ, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: read tcp 188.166.225.170:45274->45.91.116.7:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.152974028 (189h12m46.584054169s) [noise=true , address=104.36.82.23:4476 , piece_id=5T42PWSGTXMKXI5HNT47K2ZAWA6K47JV3FI23Z7TWLOY7MYQPRCA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:1foWwyGzFenUicfY7cTU5dmdDJsvXGR3YXv1rf3QRXMUco59TV, address:104.36.82.23:4476): protocol: read tcp 188.166.225.170:38786->104.36.82.23:4476: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.153016519 (189h12m46.58409666s) [noise=true , address=77.90.31.19:4845 , piece_id=VOVPPIYLWCVKXRRH6SILENB5ZPYK7PL274CAOOS2S5YGO2ZK3BCA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:1ysgMdZQcnmzQ5Vfv3RadZN8widTZKrqDnDSthxkYEUXKgRxxF, address:77.90.31.19:4845): protocol: read tcp 188.166.225.170:50836->77.90.31.19:4845: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.153062367 (189h12m46.584142498s) [noise=true , address=77.90.31.14:4689 , piece_id=74GUX7DCA2NY3O2B7C2WBBVAK4Q7BSPC3Q7UGUP63HHC5JU4U24A, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:1bzvSPximqxjmXA4xtXZzD8Xyp27KURJ3oH1zCvbPmPwaLkuhM, address:77.90.31.14:4689): protocol: read tcp 188.166.225.170:36968->77.90.31.14:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.153104217 (189h12m46.584184358s) [noise=true , address=104.36.82.4:4809 , piece_id=NNXWZXAQZRY2XQ4XSPYNFQMKN2BP7LIJSRJDPFIMMZWR5QU5Y7EQ, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:17LrYeCkrPWdiL9gGZ568PvfFct6cLVRQE4Qa92ykLgqXSfBL9, address:104.36.82.4:4809): protocol: read tcp 188.166.225.170:38082->104.36.82.4:4809: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.153144314 (189h12m46.584224445s) [noise=true , address=104.36.82.60:4344 , piece_id=UXFEVKB7PMIYGFOP43KHQSB7FWTUBCMX2JOPKZOYCSKXRW2PGAKA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK, address:104.36.82.60:4344): protocol: read tcp 188.166.225.170:56238->104.36.82.60:4344: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.153182988 (189h12m46.584263119s) [noise=true , address=77.90.31.11:4582 , piece_id=QLEM3I4QDOC4P5ALXRELK7HYJXLL3LQH3IPXOLGH22U5BQNLE5WA, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:17VVqv2A8JgKbRYuqxsdnPhomFxdBaznCHMDV21XFsCpuAFBNx, address:77.90.31.11:4582): protocol: read tcp 188.166.225.170:36122->77.90.31.11:4582: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.153220901 (189h12m46.584301032s) [noise=true , address=104.36.82.58:4298 , piece_id=DZIAHLCPRWCHJY6YLXJ27R52V5QH232NK7Q3ZK27VJOVCASOOOZQ, seg_pos={0 0}, upload=196840]: Done uploading piece. err: ecclient: upload failed (node:1MWsyGhKkDhjngUDDRvxmhQddbNnRZ1A3mpzEtjLs9ZcRC5LBz, address:104.36.82.58:4298): protocol: read tcp 188.166.225.170:40164->104.36.82.58:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:38:52.153241419 (189h12m46.58432155s) [seg_pos={0 0}, upload=196840]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Aug 16 14:38:52.382302142 (189h12m46.813382263s) [upload=196840]: Done uploading segments... Aug 16 14:38:52.382329385 (189h12m46.813409516s) [upload=196840]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1m0.231848315s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 1m3.950841565s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-16 14:38:46 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": remote error: tls: internal error Duration: 1m3.950863937s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................................X........XXX..X] duration: 30.000982944s FAIL (2025-08-16 14:38:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001004025s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............X...............X............................X.] duration: 3.409878241s FAIL (2025-08-16 14:38:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 49dfafbb94c33c8117bc19860e46bb56246facc41c7f02daffe0cf8e29c76acb != before 1be7850c2cc74e399df94f373bb6443b6a49bf63dc3bd4b7e451763927467f44 ======LOGS====== Aug 16 14:37:37.968738630 (189h11m32.399818761s) [upload=196837]: Starting upload Aug 16 14:37:37.968781391 (189h11m32.399861513s) [upload=196837]: Done starting upload Aug 16 14:37:37.968833571 (189h11m32.399913702s) [upload=196837]: Uploading segments... Aug 16 14:37:37.968862807 (189h11m32.399942938s) [upload=196837]: Got next segment. Inline: true Aug 16 14:37:37.968866314 (189h11m32.399946445s) [upload=196837]: Waiting for error group managing segments... Aug 16 14:37:37.968908495 (189h11m32.399988646s) [upload=196837]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:37:37.968914897 (189h11m32.399995038s) [upload=196837]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:37:37.968919596 (189h11m32.399999747s) [upload=196837]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:37:41.759696736 (189h11m36.190776867s) [upload=196837]: Done uploading segments... Aug 16 14:37:41.759716192 (189h11m36.190796323s) [upload=196837]: Upload finished. err: <nil> Duration:* 3.409899863s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X...............X............................X] duration: 5.532200054s FAIL (2025-08-16 14:37:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 49dfafbb94c33c8117bc19860e46bb56246facc41c7f02daffe0cf8e29c76acb != before 1be7850c2cc74e399df94f373bb6443b6a49bf63dc3bd4b7e451763927467f44 ======LOGS====== Aug 16 14:37:37.968738630 (189h11m32.399818761s) [upload=196837]: Starting upload Aug 16 14:37:37.968781391 (189h11m32.399861513s) [upload=196837]: Done starting upload Aug 16 14:37:37.968833571 (189h11m32.399913702s) [upload=196837]: Uploading segments... Aug 16 14:37:37.968862807 (189h11m32.399942938s) [upload=196837]: Got next segment. Inline: true Aug 16 14:37:37.968866314 (189h11m32.399946445s) [upload=196837]: Waiting for error group managing segments... Aug 16 14:37:37.968908495 (189h11m32.399988646s) [upload=196837]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:37:37.968914897 (189h11m32.399995038s) [upload=196837]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:37:37.968919596 (189h11m32.399999747s) [upload=196837]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:37:41.759696736 (189h11m36.190776867s) [upload=196837]: Done uploading segments... Aug 16 14:37:41.759716192 (189h11m36.190796323s) [upload=196837]: Upload finished. err: <nil> Duration:* 5.532224761s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 2.437567692s FAIL (2025-08-16 14:37:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.437599413s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XXX..] duration: 8.273687689s FAIL (2025-08-16 14:37:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ={0 0}, upload=196820]: Begin upload segment... Aug 16 14:36:23.525955069 (189h10m17.9570352s) [seg_pos={0 0}, upload=196820]: Done begin upload segment. Aug 16 14:36:23.525977080 (189h10m17.957057211s) [noise=true , address=77.90.31.16:4735 , piece_id=VUHDEB62DCUJALFBDSEUJSMIDHVPBITTRH3KJXS3A5XKTOY6BLQA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526216258 (189h10m17.957296389s) [noise=true , address=104.36.82.6:4874 , piece_id=QKXE2NNFS3OWFZXXG6VVZZDXK7ZY5RVJNFLS6XWOEDOVI3TVRRFA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526257237 (189h10m17.957337368s) [noise=true , address=77.90.31.21:4877 , piece_id=764NSONAMBNPL4U7RT5325FGPOWCD2XU4HIEIFAGFZBG3PTHSYOQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526305268 (189h10m17.957385399s) [noise=true , address=77.90.31.13:4627 , piece_id=PYGI2ORIFBAH4AYT7G2FJ3DSKSSCAABDUGBVCUQ6GQEMGSM3LG6Q, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526338433 (189h10m17.957418564s) [noise=true , address=45.91.116.6:4975 , piece_id=KJXDM2YFZAHWYDTLMKH4ZLMZHAUFXGQNJO7DQTX2ES2FUPNB6BVQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526383518 (189h10m17.957463639s) [noise=true , address=104.36.82.12:4151 , piece_id=JV4UYRZ4GM2HTYXG3WUI7UKM2W3AE7ZZ4FNSMXCAIDRUG5VQDC4A, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526413185 (189h10m17.957493316s) [noise=true , address=104.36.82.2:4730 , piece_id=MEPZAHPWMBLUG42GED2DGG6FDPTM2KC5DYKT5LLYA2N7YEOXSNCQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526443663 (189h10m17.957523794s) [noise=true , address=104.36.82.13:4187 , piece_id=KTC5LAVKKYSIRRAANJQN24H5273HCGGXXCZ2K2PCH7BUZ2ERKUUA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526488489 (189h10m17.95756862s) [noise=true , address=77.90.31.21:4878 , piece_id=RKPYQF566SUY6D47M7STHDISLSWAFDRSOQBK7H322CKLALASJSHA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526519969 (189h10m17.9576001s) [noise=true , address=104.36.82.15:4257 , piece_id=RSUG5KCOQ2B2VQPRH6GUQ5PSKY2VD2BJNQGDFQNXW2QXSJUZJZUA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526562952 (189h10m17.957643083s) [noise=true , address=104.36.82.4:4802 , piece_id=5GR33S3YX5YUORL5RJXELMNE4DXAKZ6DFTCRHWTF4UFF2GBUXHZQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526596105 (189h10m17.957676236s) [noise=true , address=45.91.116.12:4227 , piece_id=3L5DYJSBR56VXW5OJ6CYLY6NSFOMZUGR2FLNTDUBHTFMMWN6B7MA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526685316 (189h10m17.957765447s) [noise=true , address=104.36.82.3:4767 , piece_id=QRMHSTW2PZK5NFTGL7ZEH2EGOQXWUICL52ISHPDF75FNKMVWLXGA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526733959 (189h10m17.95781409s) [noise=true , address=45.91.116.4:4938 , piece_id=2CC7MY3GNPTTCNKFYSRMMTQO3Z3RYGVG26ADTZH4KTDKZWATR76A, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.527058720 (189h10m17.958138852s) [noise=true , address=104.36.82.8:4972 , piece_id=N7UO7XL6W7XPEEGPXYWJ6DPMJ22WYCIBJCQJTOHX4CUDIG3B7JAA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.527520644 (189h10m17.958600776s) [noise=true , address=104.36.82.11:4115 , piece_id=CZPBOG3IUHUIWTNNYPNZTYHEINRK2RHYMTXL72DM3M3ZIEBUPVNA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:25.323993740 (189h10m19.755073871s) [upload=196820]: Next returned nil segment Aug 16 14:36:25.324010502 (189h10m19.755090633s) [upload=196820]: Waiting for error group managing segments... Aug 16 14:36:25.695412579 (189h10m20.12649271s) [noise=true , address=104.36.82.11:4115 , piece_id=CZPBOG3IUHUIWTNNYPNZTYHEINRK2RHYMTXL72DM3M3ZIEBUPVNA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.796201608 (189h10m20.227281729s) [noise=true , address=104.36.82.6:4874 , piece_id=QKXE2NNFS3OWFZXXG6VVZZDXK7ZY5RVJNFLS6XWOEDOVI3TVRRFA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.812935036 (189h10m20.244015167s) [noise=true , address=104.36.82.3:4767 , piece_id=QRMHSTW2PZK5NFTGL7ZEH2EGOQXWUICL52ISHPDF75FNKMVWLXGA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.951639892 (189h10m20.382720043s) [noise=true , address=104.36.82.2:4730 , piece_id=MEPZAHPWMBLUG42GED2DGG6FDPTM2KC5DYKT5LLYA2N7YEOXSNCQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:26.325196452 (189h10m20.756276583s) [noise=true , address=104.36.82.4:4802 , piece_id=5GR33S3YX5YUORL5RJXELMNE4DXAKZ6DFTCRHWTF4UFF2GBUXHZQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:26.455717299 (189h10m20.88679743s) [noise=true , address=45.91.116.12:4227 , piece_id=3L5DYJSBR56VXW5OJ6CYLY6NSFOMZUGR2FLNTDUBHTFMMWN6B7MA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:29.846714183 (189h10m24.277794314s) [noise=true , address=77.90.31.21:4877 , piece_id=764NSONAMBNPL4U7RT5325FGPOWCD2XU4HIEIFAGFZBG3PTHSYOQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:52.425338413 (189h10m46.856418544s) [noise=true , address=77.90.31.16:4735 , piece_id=VUHDEB62DCUJALFBDSEUJSMIDHVPBITTRH3KJXS3A5XKTOY6BLQA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425385362 (189h10m46.856465493s) [noise=true , address=77.90.31.13:4627 , piece_id=PYGI2ORIFBAH4AYT7G2FJ3DSKSSCAABDUGBVCUQ6GQEMGSM3LG6Q, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:45772->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425425078 (189h10m46.856505209s) [noise=true , address=104.36.82.13:4187 , piece_id=KTC5LAVKKYSIRRAANJQN24H5273HCGGXXCZ2K2PCH7BUZ2ERKUUA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: read tcp 188.166.225.170:49434->104.36.82.13:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425450026 (189h10m46.856530157s) [noise=true , address=104.36.82.12:4151 , piece_id=JV4UYRZ4GM2HTYXG3WUI7UKM2W3AE7ZZ4FNSMXCAIDRUG5VQDC4A, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection Aug 16 14:36:52.425457029 (189h10m46.85653716s) [noise=true , address=104.36.82.8:4972 , piece_id=N7UO7XL6W7XPEEGPXYWJ6DPMJ22WYCIBJCQJTOHX4CUDIG3B7JAA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: read tcp 188.166.225.170:43314->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425480404 (189h10m46.856560525s) [noise=true , address=45.91.116.4:4938 , piece_id=2CC7MY3GNPTTCNKFYSRMMTQO3Z3RYGVG26ADTZH4KTDKZWATR76A, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425511253 (189h10m46.856591374s) [noise=true , address=104.36.82.15:4257 , piece_id=RSUG5KCOQ2B2VQPRH6GUQ5PSKY2VD2BJNQGDFQNXW2QXSJUZJZUA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: read tcp 188.166.225.170:47184->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425531041 (189h10m46.856611172s) [noise=true , address=45.91.116.6:4975 , piece_id=KJXDM2YFZAHWYDTLMKH4ZLMZHAUFXGQNJO7DQTX2ES2FUPNB6BVQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 188.166.225.170:56156->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425546661 (189h10m46.856626792s) [noise=true , address=77.90.31.21:4878 , piece_id=RKPYQF566SUY6D47M7STHDISLSWAFDRSOQBK7H322CKLALASJSHA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: read tcp 188.166.225.170:58454->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425566118 (189h10m46.856646249s) [seg_pos={0 0}, upload=196820]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 16 14:36:52.651165147 (189h10m47.082245278s) [upload=196820]: Done uploading segments... Aug 16 14:36:52.651195295 (189h10m47.082275426s) [upload=196820]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 8.273738636s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X...X...........X................................X.X.] duration: 19.330053211s FAIL (2025-08-16 14:37:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 16 14:36:03.135249649 (189h9m57.56632978s) [upload=196813]: Starting upload Aug 16 14:36:03.135297891 (189h9m57.566378012s) [upload=196813]: Done starting upload Aug 16 14:36:03.135620919 (189h9m57.56670105s) [upload=196813]: Uploading segments... Aug 16 14:36:03.135678419 (189h9m57.56675855s) [upload=196813]: Got next segment. Inline: false Aug 16 14:36:03.135710451 (189h9m57.566790582s) [upload=196813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:36:03.135716702 (189h9m57.566796844s) [upload=196813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 14:36:04.749478778 (189h9m59.180558909s) [seg_pos={0 0}, upload=196813]: Begin upload segment... Aug 16 14:36:04.749526007 (189h9m59.180606138s) [seg_pos={0 0}, upload=196813]: Done begin upload segment. Aug 16 14:36:04.749548320 (189h9m59.180628451s) [noise=true , address=45.91.116.12:4228 , piece_id=L67L3AQBGP23ASMLGRJO62US5LCX2GFCVDXTJTQRROI6EZG37L6A, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.749632441 (189h9m59.180712582s) [noise=true , address=104.36.82.23:4469 , piece_id=XHSUNC4UEGRODZKTWGRV73FLKZYZWUFFNM6CTPLZQDX4QW2PXLQQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.749993482 (189h9m59.181073603s) [noise=true , address=104.36.82.36:4625 , piece_id=MXZ3TO2XISX6MA2ENPWZ32LHUNNVCQTUOMI6DJ4HB2NOZ2KE4D5Q, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750161975 (189h9m59.181242106s) [noise=true , address=104.36.82.60:4349 , piece_id=IJCYHM5MYLRA7TOLJNETCJSZFHWOAR6JCR6Q72RHP5AEEGNHOB7A, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750271014 (189h9m59.181351145s) [noise=true , address=104.36.82.59:4329 , piece_id=UOE7VZCTSUWD5MLU2LIWD7SRBWSLASJMZ4IKDLXOR7YGXVQQ7KUQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750563362 (189h9m59.181643494s) [noise=true , address=77.90.31.11:4587 , piece_id=SLO6F3YWTWXYYWCIWRFF6GOBMGJIDCP5NXXJB5O3MJBOY4ZVW2SQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750701817 (189h9m59.181781948s) [noise=true , address=104.36.82.3:4796 , piece_id=RIEOLGP3M25DJ33NHQPG3DNM2B3OJNIVLZNTVGMJB7WR5HTVKVUQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750815966 (189h9m59.181896097s) [noise=true , address=104.36.82.35:4607 , piece_id=54WQVBZDWDKCHBX4UAFY3ZVJO45AS6JC25HJR5IDVCPFTDSLUJRA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750964661 (189h9m59.182044792s) [noise=true , address=104.36.82.44:4903 , piece_id=QN4XEOXMGURLCM3T4HBJETQZM5YN3VID6C7DQBVGDXSMZRGZOAAA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751049513 (189h9m59.182129644s) [noise=true , address=79.127.201.215:4350 , piece_id=OPZJ2JUSRQ4D32O65OFM2MYMVYWUNTBLG4LMJXPXJVE3YOUKEOXQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751196594 (189h9m59.182276725s) [noise=true , address=104.36.82.38:4723 , piece_id=UMW27I5M2MH6SY5JQ4TS4SDZECXZBSUHTTVHQRFE5SKRCEHHFVEA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751275876 (189h9m59.182356007s) [noise=true , address=104.36.82.5:4850 , piece_id=IQ4LG3SLPDVGVT4YCV7FUIULNYB7N5E35UH57ETUJZZCI6MTOD5A, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751429781 (189h9m59.182509912s) [noise=true , address=104.36.82.18:4383 , piece_id=UAYV6FRVQPWIDUW5T63YKEI3FC2OV6B4VAYTCWMN4SF65RDXZROA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751503452 (189h9m59.182583583s) [noise=true , address=79.127.205.246:4282 , piece_id=V3GWKAZUIJ52C4TU4E6A5Q53XU4PJ3Y2SISJHI55NUVVYAVH4RLA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751656224 (189h9m59.182736355s) [noise=true , address=79.127.205.233:4627 , piece_id=FPYGWNYKUPIVIEAAUJOB36FVLI3VX7LZJVQ34DUNBJ7VOEWZXVWQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751737900 (189h9m59.182818031s) [noise=true , address=79.127.219.48:4094 , piece_id=BYEVT7C74YQYRQCCYGVU2IEZD4PNR2FOXTVU3ZVX745A62GAEBEQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751868079 (189h9m59.18294821s) [noise=true , address=77.90.31.21:4899 , piece_id=QKPKF6V5YYY7TUSRCXC7Q3CARIBEKYLWKJQKOQQ5SZFKJANREHMA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751939485 (189h9m59.183019616s) [noise=true , address=45.91.116.7:4017 , piece_id=FGH56FXD2IGTO3KRQZF6BXK6X7TH2Q2LDJPVM76IQMR5CIFETTUQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:05.880070272 (189h10m0.311150403s) [upload=196813]: Next returned nil segment Aug 16 14:36:05.880094218 (189h10m0.311174349s) [upload=196813]: Waiting for error group managing segments... Aug 16 14:36:06.165650899 (189h10m0.59673102s) [noise=true , address=79.127.205.233:4627 , piece_id=FPYGWNYKUPIVIEAAUJOB36FVLI3VX7LZJVQ34DUNBJ7VOEWZXVWQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:06.226880806 (189h10m0.657960937s) [noise=true , address=79.127.205.246:4282 , piece_id=V3GWKAZUIJ52C4TU4E6A5Q53XU4PJ3Y2SISJHI55NUVVYAVH4RLA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:06.799190127 (189h10m1.230270258s) [noise=true , address=79.127.219.48:4094 , piece_id=BYEVT7C74YQYRQCCYGVU2IEZD4PNR2FOXTVU3ZVX745A62GAEBEQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:07.222654489 (189h10m1.65373462s) [noise=true , address=104.36.82.44:4903 , piece_id=QN4XEOXMGURLCM3T4HBJETQZM5YN3VID6C7DQBVGDXSMZRGZOAAA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:09.318411372 (189h10m3.749491503s) [noise=true , address=104.36.82.23:4469 , piece_id=XHSUNC4UEGRODZKTWGRV73FLKZYZWUFFNM6CTPLZQDX4QW2PXLQQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:11.213186618 (189h10m5.644266749s) [noise=true , address=104.36.82.38:4723 , piece_id=UMW27I5M2MH6SY5JQ4TS4SDZECXZBSUHTTVHQRFE5SKRCEHHFVEA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:16.263298925 (189h10m10.694379056s) [noise=true , address=104.36.82.35:4607 , piece_id=54WQVBZDWDKCHBX4UAFY3ZVJO45AS6JC25HJR5IDVCPFTDSLUJRA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:52.031156178 (189h10m46.462236319s) [noise=true , address=79.127.201.215:4350 , piece_id=OPZJ2JUSRQ4D32O65OFM2MYMVYWUNTBLG4LMJXPXJVE3YOUKEOXQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:58.068118758 (189h10m52.499198889s) [noise=true , address=104.36.82.5:4850 , piece_id=IQ4LG3SLPDVGVT4YCV7FUIULNYB7N5E35UH57ETUJZZCI6MTOD5A, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:59.896232646 (189h10m54.327312777s) [noise=true , address=104.36.82.18:4383 , piece_id=UAYV6FRVQPWIDUW5T63YKEI3FC2OV6B4VAYTCWMN4SF65RDXZROA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:37:02.484340649 (189h10m56.91542078s) [noise=true , address=45.91.116.7:4017 , piece_id=FGH56FXD2IGTO3KRQZF6BXK6X7TH2Q2LDJPVM76IQMR5CIFETTUQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:1XLSYete7Dj3VNt24BbP5szLp35H6F6urWr8tRAdYy1Ms1Gpj7, address:45.91.116.7:4017): protocol: read tcp 188.166.225.170:54908->45.91.116.7:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484409541 (189h10m56.915489672s) [noise=true , address=77.90.31.21:4899 , piece_id=QKPKF6V5YYY7TUSRCXC7Q3CARIBEKYLWKJQKOQQ5SZFKJANREHMA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:12UjFkFn2UEvr57LDorcijDCvkBkkKitgZEFx6Yqfk5gMStQvJG, address:77.90.31.21:4899): protocol: rpc: dial tcp 77.90.31.21:4899: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484472301 (189h10m56.915552433s) [noise=true , address=45.91.116.12:4228 , piece_id=L67L3AQBGP23ASMLGRJO62US5LCX2GFCVDXTJTQRROI6EZG37L6A, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, address:45.91.116.12:4228): protocol: write tcp 188.166.225.170:43618->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484538267 (189h10m56.915618398s) [noise=true , address=104.36.82.3:4796 , piece_id=RIEOLGP3M25DJ33NHQPG3DNM2B3OJNIVLZNTVGMJB7WR5HTVKVUQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:12nZwQ4Poz9M14ijFV6qpy5SCWEMeyHQpxN17uSBTH1ts1CFtVS, address:104.36.82.3:4796): protocol: write tcp 188.166.225.170:41142->104.36.82.3:4796: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484625053 (189h10m56.915705184s) [noise=true , address=104.36.82.59:4329 , piece_id=UOE7VZCTSUWD5MLU2LIWD7SRBWSLASJMZ4IKDLXOR7YGXVQQ7KUQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:1dGY6BG4FULjuRwnEKd9wV4cZPVJsgo32oFHKZn5BgjxVaojbk, address:104.36.82.59:4329): protocol: read tcp 188.166.225.170:50714->104.36.82.59:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484665942 (189h10m56.915746073s) [noise=true , address=104.36.82.60:4349 , piece_id=IJCYHM5MYLRA7TOLJNETCJSZFHWOAR6JCR6Q72RHP5AEEGNHOB7A, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:12CrzYmDPm95TgyE7SHJsFVbL1JUTdwMmZVge4uEkSCbeP8iBEk, address:104.36.82.60:4349): protocol: read tcp 188.166.225.170:34854->104.36.82.60:4349: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484708543 (189h10m56.915788674s) [noise=true , address=104.36.82.36:4625 , piece_id=MXZ3TO2XISX6MA2ENPWZ32LHUNNVCQTUOMI6DJ4HB2NOZ2KE4D5Q, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF, address:104.36.82.36:4625): protocol: read tcp 188.166.225.170:54032->104.36.82.36:4625: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484737819 (189h10m56.91581795s) [noise=true , address=77.90.31.11:4587 , piece_id=SLO6F3YWTWXYYWCIWRFF6GOBMGJIDCP5NXXJB5O3MJBOY4ZVW2SQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:1twkDDcpRrmR5bFhjGKKccMxC4Y35xh7tLaFxawhBmZkVWh3Bb, address:77.90.31.11:4587): protocol: read tcp 188.166.225.170:47072->77.90.31.11:4587: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484786272 (189h10m56.915866403s) [seg_pos={0 0}, upload=196813]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 16 14:37:03.747396439 (189h10m58.17847657s) [upload=196813]: Done uploading segments... Aug 16 14:37:03.747438018 (189h10m58.178518149s) [upload=196813]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 19.330069743s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 8.307353164s FAIL (2025-08-16 14:37:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ={0 0}, upload=196820]: Begin upload segment... Aug 16 14:36:23.525955069 (189h10m17.9570352s) [seg_pos={0 0}, upload=196820]: Done begin upload segment. Aug 16 14:36:23.525977080 (189h10m17.957057211s) [noise=true , address=77.90.31.16:4735 , piece_id=VUHDEB62DCUJALFBDSEUJSMIDHVPBITTRH3KJXS3A5XKTOY6BLQA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526216258 (189h10m17.957296389s) [noise=true , address=104.36.82.6:4874 , piece_id=QKXE2NNFS3OWFZXXG6VVZZDXK7ZY5RVJNFLS6XWOEDOVI3TVRRFA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526257237 (189h10m17.957337368s) [noise=true , address=77.90.31.21:4877 , piece_id=764NSONAMBNPL4U7RT5325FGPOWCD2XU4HIEIFAGFZBG3PTHSYOQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526305268 (189h10m17.957385399s) [noise=true , address=77.90.31.13:4627 , piece_id=PYGI2ORIFBAH4AYT7G2FJ3DSKSSCAABDUGBVCUQ6GQEMGSM3LG6Q, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526338433 (189h10m17.957418564s) [noise=true , address=45.91.116.6:4975 , piece_id=KJXDM2YFZAHWYDTLMKH4ZLMZHAUFXGQNJO7DQTX2ES2FUPNB6BVQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526383518 (189h10m17.957463639s) [noise=true , address=104.36.82.12:4151 , piece_id=JV4UYRZ4GM2HTYXG3WUI7UKM2W3AE7ZZ4FNSMXCAIDRUG5VQDC4A, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526413185 (189h10m17.957493316s) [noise=true , address=104.36.82.2:4730 , piece_id=MEPZAHPWMBLUG42GED2DGG6FDPTM2KC5DYKT5LLYA2N7YEOXSNCQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526443663 (189h10m17.957523794s) [noise=true , address=104.36.82.13:4187 , piece_id=KTC5LAVKKYSIRRAANJQN24H5273HCGGXXCZ2K2PCH7BUZ2ERKUUA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526488489 (189h10m17.95756862s) [noise=true , address=77.90.31.21:4878 , piece_id=RKPYQF566SUY6D47M7STHDISLSWAFDRSOQBK7H322CKLALASJSHA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526519969 (189h10m17.9576001s) [noise=true , address=104.36.82.15:4257 , piece_id=RSUG5KCOQ2B2VQPRH6GUQ5PSKY2VD2BJNQGDFQNXW2QXSJUZJZUA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526562952 (189h10m17.957643083s) [noise=true , address=104.36.82.4:4802 , piece_id=5GR33S3YX5YUORL5RJXELMNE4DXAKZ6DFTCRHWTF4UFF2GBUXHZQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526596105 (189h10m17.957676236s) [noise=true , address=45.91.116.12:4227 , piece_id=3L5DYJSBR56VXW5OJ6CYLY6NSFOMZUGR2FLNTDUBHTFMMWN6B7MA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526685316 (189h10m17.957765447s) [noise=true , address=104.36.82.3:4767 , piece_id=QRMHSTW2PZK5NFTGL7ZEH2EGOQXWUICL52ISHPDF75FNKMVWLXGA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526733959 (189h10m17.95781409s) [noise=true , address=45.91.116.4:4938 , piece_id=2CC7MY3GNPTTCNKFYSRMMTQO3Z3RYGVG26ADTZH4KTDKZWATR76A, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.527058720 (189h10m17.958138852s) [noise=true , address=104.36.82.8:4972 , piece_id=N7UO7XL6W7XPEEGPXYWJ6DPMJ22WYCIBJCQJTOHX4CUDIG3B7JAA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.527520644 (189h10m17.958600776s) [noise=true , address=104.36.82.11:4115 , piece_id=CZPBOG3IUHUIWTNNYPNZTYHEINRK2RHYMTXL72DM3M3ZIEBUPVNA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:25.323993740 (189h10m19.755073871s) [upload=196820]: Next returned nil segment Aug 16 14:36:25.324010502 (189h10m19.755090633s) [upload=196820]: Waiting for error group managing segments... Aug 16 14:36:25.695412579 (189h10m20.12649271s) [noise=true , address=104.36.82.11:4115 , piece_id=CZPBOG3IUHUIWTNNYPNZTYHEINRK2RHYMTXL72DM3M3ZIEBUPVNA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.796201608 (189h10m20.227281729s) [noise=true , address=104.36.82.6:4874 , piece_id=QKXE2NNFS3OWFZXXG6VVZZDXK7ZY5RVJNFLS6XWOEDOVI3TVRRFA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.812935036 (189h10m20.244015167s) [noise=true , address=104.36.82.3:4767 , piece_id=QRMHSTW2PZK5NFTGL7ZEH2EGOQXWUICL52ISHPDF75FNKMVWLXGA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.951639892 (189h10m20.382720043s) [noise=true , address=104.36.82.2:4730 , piece_id=MEPZAHPWMBLUG42GED2DGG6FDPTM2KC5DYKT5LLYA2N7YEOXSNCQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:26.325196452 (189h10m20.756276583s) [noise=true , address=104.36.82.4:4802 , piece_id=5GR33S3YX5YUORL5RJXELMNE4DXAKZ6DFTCRHWTF4UFF2GBUXHZQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:26.455717299 (189h10m20.88679743s) [noise=true , address=45.91.116.12:4227 , piece_id=3L5DYJSBR56VXW5OJ6CYLY6NSFOMZUGR2FLNTDUBHTFMMWN6B7MA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:29.846714183 (189h10m24.277794314s) [noise=true , address=77.90.31.21:4877 , piece_id=764NSONAMBNPL4U7RT5325FGPOWCD2XU4HIEIFAGFZBG3PTHSYOQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:52.425338413 (189h10m46.856418544s) [noise=true , address=77.90.31.16:4735 , piece_id=VUHDEB62DCUJALFBDSEUJSMIDHVPBITTRH3KJXS3A5XKTOY6BLQA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425385362 (189h10m46.856465493s) [noise=true , address=77.90.31.13:4627 , piece_id=PYGI2ORIFBAH4AYT7G2FJ3DSKSSCAABDUGBVCUQ6GQEMGSM3LG6Q, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:45772->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425425078 (189h10m46.856505209s) [noise=true , address=104.36.82.13:4187 , piece_id=KTC5LAVKKYSIRRAANJQN24H5273HCGGXXCZ2K2PCH7BUZ2ERKUUA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: read tcp 188.166.225.170:49434->104.36.82.13:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425450026 (189h10m46.856530157s) [noise=true , address=104.36.82.12:4151 , piece_id=JV4UYRZ4GM2HTYXG3WUI7UKM2W3AE7ZZ4FNSMXCAIDRUG5VQDC4A, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection Aug 16 14:36:52.425457029 (189h10m46.85653716s) [noise=true , address=104.36.82.8:4972 , piece_id=N7UO7XL6W7XPEEGPXYWJ6DPMJ22WYCIBJCQJTOHX4CUDIG3B7JAA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: read tcp 188.166.225.170:43314->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425480404 (189h10m46.856560525s) [noise=true , address=45.91.116.4:4938 , piece_id=2CC7MY3GNPTTCNKFYSRMMTQO3Z3RYGVG26ADTZH4KTDKZWATR76A, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425511253 (189h10m46.856591374s) [noise=true , address=104.36.82.15:4257 , piece_id=RSUG5KCOQ2B2VQPRH6GUQ5PSKY2VD2BJNQGDFQNXW2QXSJUZJZUA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: read tcp 188.166.225.170:47184->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425531041 (189h10m46.856611172s) [noise=true , address=45.91.116.6:4975 , piece_id=KJXDM2YFZAHWYDTLMKH4ZLMZHAUFXGQNJO7DQTX2ES2FUPNB6BVQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 188.166.225.170:56156->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425546661 (189h10m46.856626792s) [noise=true , address=77.90.31.21:4878 , piece_id=RKPYQF566SUY6D47M7STHDISLSWAFDRSOQBK7H322CKLALASJSHA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: read tcp 188.166.225.170:58454->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425566118 (189h10m46.856646249s) [seg_pos={0 0}, upload=196820]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 16 14:36:52.651165147 (189h10m47.082245278s) [upload=196820]: Done uploading segments... Aug 16 14:36:52.651195295 (189h10m47.082275426s) [upload=196820]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 8.307375277s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................X........XXX..] duration: 6.26402464s FAIL (2025-08-16 14:37:08 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 6.264048336s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X.X.] duration: 7.591493287s FAIL (2025-08-16 14:37:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== OUI3IT7EKL3MOA, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.252607492 (189h9m50.683687623s) [noise=true , address=104.36.82.23:4480 , piece_id=VJM4NSJJSUT3LTBJKGPIAMX6M535K3A2P5F7KXTYZRZ3BOJNXZAQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.252772819 (189h9m50.68385295s) [noise=true , address=104.36.82.51:4080 , piece_id=RCIOK4XKVFP7S2AMNZP4F7YPBR5TCDD4GGUFM5MVTYCNAF5QYMJQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.253037464 (189h9m50.684117595s) [noise=true , address=77.90.31.21:4908 , piece_id=QRGF7A5AJBRUUDDLO6JJP6HTTG7SYHPAJ6FPQZA2DB7HUNQ6W45Q, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.253282814 (189h9m50.684362935s) [noise=true , address=45.91.116.7:4016 , piece_id=MKKBUFHMLBQNAIYGUX4OJTIADPJAPAEYX7FMTRGTZ4PB4BNTCM4A, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.253413735 (189h9m50.684493866s) [noise=true , address=104.36.82.3:4780 , piece_id=L5YK7IRTY5C6CQSOWP4BPAIMTYCEOXEKV52QZIXWN4EVVQVTVFZQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.253631742 (189h9m50.684711873s) [noise=true , address=77.90.31.12:4597 , piece_id=KQCCRA66J4QVFVGK2VON5CMI7N2RJRI2ZQ3J2JW4IOO46JYNAUCQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.254068468 (189h9m50.685148599s) [noise=true , address=104.36.82.2:4760 , piece_id=G3SIGIQDRVULAAISXD6FRBUTWCSCMSYUGMJKRVQ5EEN4DV4Y2DWQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.254878557 (189h9m50.685958688s) [noise=true , address=104.36.82.37:4687 , piece_id=BNHBEJPEPGGFLLLJDYAWUZBILPWWQCXMO24CJAIDELPYCOT3YPRQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.255597594 (189h9m50.686677725s) [noise=true , address=104.36.82.38:4698 , piece_id=5M5EX5PO2BX35K7PKSE7K54DOO2ZZUUQTYLEPM22YHMETC4PN2PA, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.256229352 (189h9m50.687309483s) [noise=true , address=66.23.193.175:4559 , piece_id=7XSTTGKFHKLUOA5MLLGAKPNOLC5L7SH4ADKR4F7VXEUCBVV6QHFQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.257089047 (189h9m50.688169178s) [noise=true , address=79.127.201.214:4309 , piece_id=Q37SMGOPPQ3SHA3OODGFG5JKB7ZOL6FO7SH2QQG4O2CST5LZLVYA, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.257514822 (189h9m50.688594953s) [noise=true , address=104.36.82.60:4341 , piece_id=B7VRPB4BIFJPGYUH5DTY2ZFNQX7D2OGSEI4ZZSL7ZJ7GLDLWVLYQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:57.388772618 (189h9m51.819852749s) [upload=196811]: Next returned nil segment Aug 16 14:35:57.388793118 (189h9m51.819873249s) [upload=196811]: Waiting for error group managing segments... Aug 16 14:35:57.704974797 (189h9m52.136054928s) [noise=true , address=66.23.193.175:4559 , piece_id=7XSTTGKFHKLUOA5MLLGAKPNOLC5L7SH4ADKR4F7VXEUCBVV6QHFQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:35:58.749330278 (189h9m53.180410399s) [noise=true , address=104.36.82.60:4341 , piece_id=B7VRPB4BIFJPGYUH5DTY2ZFNQX7D2OGSEI4ZZSL7ZJ7GLDLWVLYQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:35:58.844329108 (189h9m53.275409239s) [noise=true , address=104.36.82.44:4877 , piece_id=7XQM3JXHEJDOGA7VUKCQ3YL6R7QOA3ATKHTG65UJ7DNBRJS22DNQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:35:59.085549518 (189h9m53.516629639s) [noise=true , address=104.36.82.51:4080 , piece_id=RCIOK4XKVFP7S2AMNZP4F7YPBR5TCDD4GGUFM5MVTYCNAF5QYMJQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:01.982810121 (189h9m56.413890262s) [noise=true , address=104.36.82.38:4698 , piece_id=5M5EX5PO2BX35K7PKSE7K54DOO2ZZUUQTYLEPM22YHMETC4PN2PA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:14.358553013 (189h10m8.789633134s) [noise=true , address=104.36.82.2:4760 , piece_id=G3SIGIQDRVULAAISXD6FRBUTWCSCMSYUGMJKRVQ5EEN4DV4Y2DWQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:47.398775401 (189h10m41.829855532s) [noise=true , address=79.127.201.214:4309 , piece_id=Q37SMGOPPQ3SHA3OODGFG5JKB7ZOL6FO7SH2QQG4O2CST5LZLVYA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:51.783121710 (189h10m46.214201851s) [noise=true , address=79.127.201.215:4349 , piece_id=WFCPXTQZV2V3743AX34R4J3CE24BKO62QMCF23T7UXGLZS5XHK4Q, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:55.095820859 (189h10m49.52690099s) [noise=true , address=45.91.116.7:4016 , piece_id=MKKBUFHMLBQNAIYGUX4OJTIADPJAPAEYX7FMTRGTZ4PB4BNTCM4A, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: write tcp 188.166.225.170:41250->45.91.116.7:4016: use of closed network connection; write tcp 188.166.225.170:41250->45.91.116.7:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41250->45.91.116.7:4016: use of closed network connection Aug 16 14:36:55.095844455 (189h10m49.526924586s) [noise=true , address=104.36.82.5:4861 , piece_id=LTHKBT73MU53FQXE6JVBUHO5FVYGRLIPLLHN6WAGSFDBO3KZPKOA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 188.166.225.170:53682->104.36.82.5:4861: use of closed network connection; write tcp 188.166.225.170:53682->104.36.82.5:4861: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53682->104.36.82.5:4861: use of closed network connection Aug 16 14:36:55.095884270 (189h10m49.526964402s) [noise=true , address=104.36.82.3:4780 , piece_id=L5YK7IRTY5C6CQSOWP4BPAIMTYCEOXEKV52QZIXWN4EVVQVTVFZQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM, address:104.36.82.3:4780): protocol: rpc: dial tcp 104.36.82.3:4780: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.095898648 (189h10m49.526978779s) [noise=true , address=77.90.31.15:4729 , piece_id=UZQDC3VDA5DPWNRRBLYVTZGKL5LFVLTOC3GECXOUI3IT7EKL3MOA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1ArQdwHzHSm7c2QZkTt1kQsfzocvCNG2D9yoQgnmbS7HYLCXhy, address:77.90.31.15:4729): protocol: rpc: dial tcp 77.90.31.15:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.095931241 (189h10m49.527011372s) [noise=true , address=104.36.82.23:4480 , piece_id=VJM4NSJJSUT3LTBJKGPIAMX6M535K3A2P5F7KXTYZRZ3BOJNXZAQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1qQhWAdN83i1jMTYByNQ8SWGXQaXRgBGezkLTf1BTwLyKskjjU, address:104.36.82.23:4480): protocol: rpc: dial tcp 104.36.82.23:4480: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.095983661 (189h10m49.527063782s) [noise=true , address=77.90.31.12:4597 , piece_id=KQCCRA66J4QVFVGK2VON5CMI7N2RJRI2ZQ3J2JW4IOO46JYNAUCQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:127QMaS39fDFZixRP6gVMQkZMefSxLo3ToPctZuDmaARY6nB3H9, address:77.90.31.12:4597): protocol: write tcp 188.166.225.170:50376->77.90.31.12:4597: use of closed network connection; write tcp 188.166.225.170:50376->77.90.31.12:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50376->77.90.31.12:4597: use of closed network connection Aug 16 14:36:55.096009180 (189h10m49.527089301s) [noise=true , address=77.90.31.21:4908 , piece_id=QRGF7A5AJBRUUDDLO6JJP6HTTG7SYHPAJ6FPQZA2DB7HUNQ6W45Q, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:13hCBBV3cxyTWCAuG8QGGAoJRb6S8Ay23HpXYyBgRX4HZSSpga, address:77.90.31.21:4908): protocol: write tcp 188.166.225.170:45214->77.90.31.21:4908: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.096044277 (189h10m49.527124408s) [noise=true , address=77.90.31.13:4639 , piece_id=FJWIJ66PFT3BQ3BA55UBPSBF7QLX77Y7RD3OEDWYQLTXJYL6I6VA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:12N9N1pcaVn3nPh1WY7uktz8Z7fwJGqnTdVwZMAW5SVrqvRA8VV, address:77.90.31.13:4639): protocol: write tcp 188.166.225.170:58274->77.90.31.13:4639: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.096091648 (189h10m49.527171799s) [noise=true , address=104.36.82.37:4687 , piece_id=BNHBEJPEPGGFLLLJDYAWUZBILPWWQCXMO24CJAIDELPYCOT3YPRQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:12JZW19SfSwU95frntTgqDhiam1Rva1U8V7yfQkqtKYWntJ24Qf, address:104.36.82.37:4687): protocol: read tcp 188.166.225.170:50748->104.36.82.37:4687: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.096107457 (189h10m49.527187578s) [noise=true , address=45.91.116.5:4306 , piece_id=IWXOBJ7G3OY4BYIPPV35S3YH553UXCFSS75Q7KSUVKUHRTC7Z4IA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1QuFvRW9wGKdKRXPK8UdR8uqTm53CnHaF9CsUz2bUh2Ptw9r1F, address:45.91.116.5:4306): protocol: read tcp 188.166.225.170:48096->45.91.116.5:4306: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.096136523 (189h10m49.527216654s) [seg_pos={0 0}, upload=196811]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Aug 16 14:36:55.320533012 (189h10m49.751613143s) [upload=196811]: Done uploading segments... Aug 16 14:36:55.320556706 (189h10m49.751636827s) [upload=196811]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 7.591514878s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.......X...X...........X................................X.X] duration: 1m1.265129172s FAIL (2025-08-16 14:37:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 16 14:36:03.135249649 (189h9m57.56632978s) [upload=196813]: Starting upload Aug 16 14:36:03.135297891 (189h9m57.566378012s) [upload=196813]: Done starting upload Aug 16 14:36:03.135620919 (189h9m57.56670105s) [upload=196813]: Uploading segments... Aug 16 14:36:03.135678419 (189h9m57.56675855s) [upload=196813]: Got next segment. Inline: false Aug 16 14:36:03.135710451 (189h9m57.566790582s) [upload=196813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:36:03.135716702 (189h9m57.566796844s) [upload=196813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 14:36:04.749478778 (189h9m59.180558909s) [seg_pos={0 0}, upload=196813]: Begin upload segment... Aug 16 14:36:04.749526007 (189h9m59.180606138s) [seg_pos={0 0}, upload=196813]: Done begin upload segment. Aug 16 14:36:04.749548320 (189h9m59.180628451s) [noise=true , address=45.91.116.12:4228 , piece_id=L67L3AQBGP23ASMLGRJO62US5LCX2GFCVDXTJTQRROI6EZG37L6A, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.749632441 (189h9m59.180712582s) [noise=true , address=104.36.82.23:4469 , piece_id=XHSUNC4UEGRODZKTWGRV73FLKZYZWUFFNM6CTPLZQDX4QW2PXLQQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.749993482 (189h9m59.181073603s) [noise=true , address=104.36.82.36:4625 , piece_id=MXZ3TO2XISX6MA2ENPWZ32LHUNNVCQTUOMI6DJ4HB2NOZ2KE4D5Q, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750161975 (189h9m59.181242106s) [noise=true , address=104.36.82.60:4349 , piece_id=IJCYHM5MYLRA7TOLJNETCJSZFHWOAR6JCR6Q72RHP5AEEGNHOB7A, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750271014 (189h9m59.181351145s) [noise=true , address=104.36.82.59:4329 , piece_id=UOE7VZCTSUWD5MLU2LIWD7SRBWSLASJMZ4IKDLXOR7YGXVQQ7KUQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750563362 (189h9m59.181643494s) [noise=true , address=77.90.31.11:4587 , piece_id=SLO6F3YWTWXYYWCIWRFF6GOBMGJIDCP5NXXJB5O3MJBOY4ZVW2SQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750701817 (189h9m59.181781948s) [noise=true , address=104.36.82.3:4796 , piece_id=RIEOLGP3M25DJ33NHQPG3DNM2B3OJNIVLZNTVGMJB7WR5HTVKVUQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750815966 (189h9m59.181896097s) [noise=true , address=104.36.82.35:4607 , piece_id=54WQVBZDWDKCHBX4UAFY3ZVJO45AS6JC25HJR5IDVCPFTDSLUJRA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.750964661 (189h9m59.182044792s) [noise=true , address=104.36.82.44:4903 , piece_id=QN4XEOXMGURLCM3T4HBJETQZM5YN3VID6C7DQBVGDXSMZRGZOAAA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751049513 (189h9m59.182129644s) [noise=true , address=79.127.201.215:4350 , piece_id=OPZJ2JUSRQ4D32O65OFM2MYMVYWUNTBLG4LMJXPXJVE3YOUKEOXQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751196594 (189h9m59.182276725s) [noise=true , address=104.36.82.38:4723 , piece_id=UMW27I5M2MH6SY5JQ4TS4SDZECXZBSUHTTVHQRFE5SKRCEHHFVEA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751275876 (189h9m59.182356007s) [noise=true , address=104.36.82.5:4850 , piece_id=IQ4LG3SLPDVGVT4YCV7FUIULNYB7N5E35UH57ETUJZZCI6MTOD5A, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751429781 (189h9m59.182509912s) [noise=true , address=104.36.82.18:4383 , piece_id=UAYV6FRVQPWIDUW5T63YKEI3FC2OV6B4VAYTCWMN4SF65RDXZROA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751503452 (189h9m59.182583583s) [noise=true , address=79.127.205.246:4282 , piece_id=V3GWKAZUIJ52C4TU4E6A5Q53XU4PJ3Y2SISJHI55NUVVYAVH4RLA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751656224 (189h9m59.182736355s) [noise=true , address=79.127.205.233:4627 , piece_id=FPYGWNYKUPIVIEAAUJOB36FVLI3VX7LZJVQ34DUNBJ7VOEWZXVWQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751737900 (189h9m59.182818031s) [noise=true , address=79.127.219.48:4094 , piece_id=BYEVT7C74YQYRQCCYGVU2IEZD4PNR2FOXTVU3ZVX745A62GAEBEQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751868079 (189h9m59.18294821s) [noise=true , address=77.90.31.21:4899 , piece_id=QKPKF6V5YYY7TUSRCXC7Q3CARIBEKYLWKJQKOQQ5SZFKJANREHMA, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:04.751939485 (189h9m59.183019616s) [noise=true , address=45.91.116.7:4017 , piece_id=FGH56FXD2IGTO3KRQZF6BXK6X7TH2Q2LDJPVM76IQMR5CIFETTUQ, seg_pos={0 0}, upload=196813]: Uploading piece... Aug 16 14:36:05.880070272 (189h10m0.311150403s) [upload=196813]: Next returned nil segment Aug 16 14:36:05.880094218 (189h10m0.311174349s) [upload=196813]: Waiting for error group managing segments... Aug 16 14:36:06.165650899 (189h10m0.59673102s) [noise=true , address=79.127.205.233:4627 , piece_id=FPYGWNYKUPIVIEAAUJOB36FVLI3VX7LZJVQ34DUNBJ7VOEWZXVWQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:06.226880806 (189h10m0.657960937s) [noise=true , address=79.127.205.246:4282 , piece_id=V3GWKAZUIJ52C4TU4E6A5Q53XU4PJ3Y2SISJHI55NUVVYAVH4RLA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:06.799190127 (189h10m1.230270258s) [noise=true , address=79.127.219.48:4094 , piece_id=BYEVT7C74YQYRQCCYGVU2IEZD4PNR2FOXTVU3ZVX745A62GAEBEQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:07.222654489 (189h10m1.65373462s) [noise=true , address=104.36.82.44:4903 , piece_id=QN4XEOXMGURLCM3T4HBJETQZM5YN3VID6C7DQBVGDXSMZRGZOAAA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:09.318411372 (189h10m3.749491503s) [noise=true , address=104.36.82.23:4469 , piece_id=XHSUNC4UEGRODZKTWGRV73FLKZYZWUFFNM6CTPLZQDX4QW2PXLQQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:11.213186618 (189h10m5.644266749s) [noise=true , address=104.36.82.38:4723 , piece_id=UMW27I5M2MH6SY5JQ4TS4SDZECXZBSUHTTVHQRFE5SKRCEHHFVEA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:16.263298925 (189h10m10.694379056s) [noise=true , address=104.36.82.35:4607 , piece_id=54WQVBZDWDKCHBX4UAFY3ZVJO45AS6JC25HJR5IDVCPFTDSLUJRA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:52.031156178 (189h10m46.462236319s) [noise=true , address=79.127.201.215:4350 , piece_id=OPZJ2JUSRQ4D32O65OFM2MYMVYWUNTBLG4LMJXPXJVE3YOUKEOXQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:58.068118758 (189h10m52.499198889s) [noise=true , address=104.36.82.5:4850 , piece_id=IQ4LG3SLPDVGVT4YCV7FUIULNYB7N5E35UH57ETUJZZCI6MTOD5A, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:36:59.896232646 (189h10m54.327312777s) [noise=true , address=104.36.82.18:4383 , piece_id=UAYV6FRVQPWIDUW5T63YKEI3FC2OV6B4VAYTCWMN4SF65RDXZROA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: <nil> Aug 16 14:37:02.484340649 (189h10m56.91542078s) [noise=true , address=45.91.116.7:4017 , piece_id=FGH56FXD2IGTO3KRQZF6BXK6X7TH2Q2LDJPVM76IQMR5CIFETTUQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:1XLSYete7Dj3VNt24BbP5szLp35H6F6urWr8tRAdYy1Ms1Gpj7, address:45.91.116.7:4017): protocol: read tcp 188.166.225.170:54908->45.91.116.7:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484409541 (189h10m56.915489672s) [noise=true , address=77.90.31.21:4899 , piece_id=QKPKF6V5YYY7TUSRCXC7Q3CARIBEKYLWKJQKOQQ5SZFKJANREHMA, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:12UjFkFn2UEvr57LDorcijDCvkBkkKitgZEFx6Yqfk5gMStQvJG, address:77.90.31.21:4899): protocol: rpc: dial tcp 77.90.31.21:4899: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484472301 (189h10m56.915552433s) [noise=true , address=45.91.116.12:4228 , piece_id=L67L3AQBGP23ASMLGRJO62US5LCX2GFCVDXTJTQRROI6EZG37L6A, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, address:45.91.116.12:4228): protocol: write tcp 188.166.225.170:43618->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484538267 (189h10m56.915618398s) [noise=true , address=104.36.82.3:4796 , piece_id=RIEOLGP3M25DJ33NHQPG3DNM2B3OJNIVLZNTVGMJB7WR5HTVKVUQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:12nZwQ4Poz9M14ijFV6qpy5SCWEMeyHQpxN17uSBTH1ts1CFtVS, address:104.36.82.3:4796): protocol: write tcp 188.166.225.170:41142->104.36.82.3:4796: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484625053 (189h10m56.915705184s) [noise=true , address=104.36.82.59:4329 , piece_id=UOE7VZCTSUWD5MLU2LIWD7SRBWSLASJMZ4IKDLXOR7YGXVQQ7KUQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:1dGY6BG4FULjuRwnEKd9wV4cZPVJsgo32oFHKZn5BgjxVaojbk, address:104.36.82.59:4329): protocol: read tcp 188.166.225.170:50714->104.36.82.59:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484665942 (189h10m56.915746073s) [noise=true , address=104.36.82.60:4349 , piece_id=IJCYHM5MYLRA7TOLJNETCJSZFHWOAR6JCR6Q72RHP5AEEGNHOB7A, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:12CrzYmDPm95TgyE7SHJsFVbL1JUTdwMmZVge4uEkSCbeP8iBEk, address:104.36.82.60:4349): protocol: read tcp 188.166.225.170:34854->104.36.82.60:4349: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484708543 (189h10m56.915788674s) [noise=true , address=104.36.82.36:4625 , piece_id=MXZ3TO2XISX6MA2ENPWZ32LHUNNVCQTUOMI6DJ4HB2NOZ2KE4D5Q, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF, address:104.36.82.36:4625): protocol: read tcp 188.166.225.170:54032->104.36.82.36:4625: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484737819 (189h10m56.91581795s) [noise=true , address=77.90.31.11:4587 , piece_id=SLO6F3YWTWXYYWCIWRFF6GOBMGJIDCP5NXXJB5O3MJBOY4ZVW2SQ, seg_pos={0 0}, upload=196813]: Done uploading piece. err: ecclient: upload failed (node:1twkDDcpRrmR5bFhjGKKccMxC4Y35xh7tLaFxawhBmZkVWh3Bb, address:77.90.31.11:4587): protocol: read tcp 188.166.225.170:47072->77.90.31.11:4587: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:37:02.484786272 (189h10m56.915866403s) [seg_pos={0 0}, upload=196813]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 16 14:37:03.747396439 (189h10m58.17847657s) [upload=196813]: Done uploading segments... Aug 16 14:37:03.747438018 (189h10m58.178518149s) [upload=196813]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m1.265155863s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................X........XXX.] duration: 30.000939754s FAIL (2025-08-16 14:37:01 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 30.000949291s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 18.514723258s FAIL (2025-08-16 14:36:57 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 18.514744287s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 1m0.225999497s FAIL (2025-08-16 14:36:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== OUI3IT7EKL3MOA, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.252607492 (189h9m50.683687623s) [noise=true , address=104.36.82.23:4480 , piece_id=VJM4NSJJSUT3LTBJKGPIAMX6M535K3A2P5F7KXTYZRZ3BOJNXZAQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.252772819 (189h9m50.68385295s) [noise=true , address=104.36.82.51:4080 , piece_id=RCIOK4XKVFP7S2AMNZP4F7YPBR5TCDD4GGUFM5MVTYCNAF5QYMJQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.253037464 (189h9m50.684117595s) [noise=true , address=77.90.31.21:4908 , piece_id=QRGF7A5AJBRUUDDLO6JJP6HTTG7SYHPAJ6FPQZA2DB7HUNQ6W45Q, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.253282814 (189h9m50.684362935s) [noise=true , address=45.91.116.7:4016 , piece_id=MKKBUFHMLBQNAIYGUX4OJTIADPJAPAEYX7FMTRGTZ4PB4BNTCM4A, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.253413735 (189h9m50.684493866s) [noise=true , address=104.36.82.3:4780 , piece_id=L5YK7IRTY5C6CQSOWP4BPAIMTYCEOXEKV52QZIXWN4EVVQVTVFZQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.253631742 (189h9m50.684711873s) [noise=true , address=77.90.31.12:4597 , piece_id=KQCCRA66J4QVFVGK2VON5CMI7N2RJRI2ZQ3J2JW4IOO46JYNAUCQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.254068468 (189h9m50.685148599s) [noise=true , address=104.36.82.2:4760 , piece_id=G3SIGIQDRVULAAISXD6FRBUTWCSCMSYUGMJKRVQ5EEN4DV4Y2DWQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.254878557 (189h9m50.685958688s) [noise=true , address=104.36.82.37:4687 , piece_id=BNHBEJPEPGGFLLLJDYAWUZBILPWWQCXMO24CJAIDELPYCOT3YPRQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.255597594 (189h9m50.686677725s) [noise=true , address=104.36.82.38:4698 , piece_id=5M5EX5PO2BX35K7PKSE7K54DOO2ZZUUQTYLEPM22YHMETC4PN2PA, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.256229352 (189h9m50.687309483s) [noise=true , address=66.23.193.175:4559 , piece_id=7XSTTGKFHKLUOA5MLLGAKPNOLC5L7SH4ADKR4F7VXEUCBVV6QHFQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.257089047 (189h9m50.688169178s) [noise=true , address=79.127.201.214:4309 , piece_id=Q37SMGOPPQ3SHA3OODGFG5JKB7ZOL6FO7SH2QQG4O2CST5LZLVYA, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:56.257514822 (189h9m50.688594953s) [noise=true , address=104.36.82.60:4341 , piece_id=B7VRPB4BIFJPGYUH5DTY2ZFNQX7D2OGSEI4ZZSL7ZJ7GLDLWVLYQ, seg_pos={0 0}, upload=196811]: Uploading piece... Aug 16 14:35:57.388772618 (189h9m51.819852749s) [upload=196811]: Next returned nil segment Aug 16 14:35:57.388793118 (189h9m51.819873249s) [upload=196811]: Waiting for error group managing segments... Aug 16 14:35:57.704974797 (189h9m52.136054928s) [noise=true , address=66.23.193.175:4559 , piece_id=7XSTTGKFHKLUOA5MLLGAKPNOLC5L7SH4ADKR4F7VXEUCBVV6QHFQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:35:58.749330278 (189h9m53.180410399s) [noise=true , address=104.36.82.60:4341 , piece_id=B7VRPB4BIFJPGYUH5DTY2ZFNQX7D2OGSEI4ZZSL7ZJ7GLDLWVLYQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:35:58.844329108 (189h9m53.275409239s) [noise=true , address=104.36.82.44:4877 , piece_id=7XQM3JXHEJDOGA7VUKCQ3YL6R7QOA3ATKHTG65UJ7DNBRJS22DNQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:35:59.085549518 (189h9m53.516629639s) [noise=true , address=104.36.82.51:4080 , piece_id=RCIOK4XKVFP7S2AMNZP4F7YPBR5TCDD4GGUFM5MVTYCNAF5QYMJQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:01.982810121 (189h9m56.413890262s) [noise=true , address=104.36.82.38:4698 , piece_id=5M5EX5PO2BX35K7PKSE7K54DOO2ZZUUQTYLEPM22YHMETC4PN2PA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:14.358553013 (189h10m8.789633134s) [noise=true , address=104.36.82.2:4760 , piece_id=G3SIGIQDRVULAAISXD6FRBUTWCSCMSYUGMJKRVQ5EEN4DV4Y2DWQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:47.398775401 (189h10m41.829855532s) [noise=true , address=79.127.201.214:4309 , piece_id=Q37SMGOPPQ3SHA3OODGFG5JKB7ZOL6FO7SH2QQG4O2CST5LZLVYA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:51.783121710 (189h10m46.214201851s) [noise=true , address=79.127.201.215:4349 , piece_id=WFCPXTQZV2V3743AX34R4J3CE24BKO62QMCF23T7UXGLZS5XHK4Q, seg_pos={0 0}, upload=196811]: Done uploading piece. err: <nil> Aug 16 14:36:55.095820859 (189h10m49.52690099s) [noise=true , address=45.91.116.7:4016 , piece_id=MKKBUFHMLBQNAIYGUX4OJTIADPJAPAEYX7FMTRGTZ4PB4BNTCM4A, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1GeydXGpXPZzL65aqsaGiRQjz7NRzL9BZCUb5MEUWCAzNPMRPW, address:45.91.116.7:4016): protocol: write tcp 188.166.225.170:41250->45.91.116.7:4016: use of closed network connection; write tcp 188.166.225.170:41250->45.91.116.7:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41250->45.91.116.7:4016: use of closed network connection Aug 16 14:36:55.095844455 (189h10m49.526924586s) [noise=true , address=104.36.82.5:4861 , piece_id=LTHKBT73MU53FQXE6JVBUHO5FVYGRLIPLLHN6WAGSFDBO3KZPKOA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 188.166.225.170:53682->104.36.82.5:4861: use of closed network connection; write tcp 188.166.225.170:53682->104.36.82.5:4861: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53682->104.36.82.5:4861: use of closed network connection Aug 16 14:36:55.095884270 (189h10m49.526964402s) [noise=true , address=104.36.82.3:4780 , piece_id=L5YK7IRTY5C6CQSOWP4BPAIMTYCEOXEKV52QZIXWN4EVVQVTVFZQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1i8cJRxzLNvUYqELrufixk24oDZwDHroyG5Cfy13csTPgeajRM, address:104.36.82.3:4780): protocol: rpc: dial tcp 104.36.82.3:4780: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.095898648 (189h10m49.526978779s) [noise=true , address=77.90.31.15:4729 , piece_id=UZQDC3VDA5DPWNRRBLYVTZGKL5LFVLTOC3GECXOUI3IT7EKL3MOA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1ArQdwHzHSm7c2QZkTt1kQsfzocvCNG2D9yoQgnmbS7HYLCXhy, address:77.90.31.15:4729): protocol: rpc: dial tcp 77.90.31.15:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.095931241 (189h10m49.527011372s) [noise=true , address=104.36.82.23:4480 , piece_id=VJM4NSJJSUT3LTBJKGPIAMX6M535K3A2P5F7KXTYZRZ3BOJNXZAQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1qQhWAdN83i1jMTYByNQ8SWGXQaXRgBGezkLTf1BTwLyKskjjU, address:104.36.82.23:4480): protocol: rpc: dial tcp 104.36.82.23:4480: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.095983661 (189h10m49.527063782s) [noise=true , address=77.90.31.12:4597 , piece_id=KQCCRA66J4QVFVGK2VON5CMI7N2RJRI2ZQ3J2JW4IOO46JYNAUCQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:127QMaS39fDFZixRP6gVMQkZMefSxLo3ToPctZuDmaARY6nB3H9, address:77.90.31.12:4597): protocol: write tcp 188.166.225.170:50376->77.90.31.12:4597: use of closed network connection; write tcp 188.166.225.170:50376->77.90.31.12:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50376->77.90.31.12:4597: use of closed network connection Aug 16 14:36:55.096009180 (189h10m49.527089301s) [noise=true , address=77.90.31.21:4908 , piece_id=QRGF7A5AJBRUUDDLO6JJP6HTTG7SYHPAJ6FPQZA2DB7HUNQ6W45Q, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:13hCBBV3cxyTWCAuG8QGGAoJRb6S8Ay23HpXYyBgRX4HZSSpga, address:77.90.31.21:4908): protocol: write tcp 188.166.225.170:45214->77.90.31.21:4908: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.096044277 (189h10m49.527124408s) [noise=true , address=77.90.31.13:4639 , piece_id=FJWIJ66PFT3BQ3BA55UBPSBF7QLX77Y7RD3OEDWYQLTXJYL6I6VA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:12N9N1pcaVn3nPh1WY7uktz8Z7fwJGqnTdVwZMAW5SVrqvRA8VV, address:77.90.31.13:4639): protocol: write tcp 188.166.225.170:58274->77.90.31.13:4639: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.096091648 (189h10m49.527171799s) [noise=true , address=104.36.82.37:4687 , piece_id=BNHBEJPEPGGFLLLJDYAWUZBILPWWQCXMO24CJAIDELPYCOT3YPRQ, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:12JZW19SfSwU95frntTgqDhiam1Rva1U8V7yfQkqtKYWntJ24Qf, address:104.36.82.37:4687): protocol: read tcp 188.166.225.170:50748->104.36.82.37:4687: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.096107457 (189h10m49.527187578s) [noise=true , address=45.91.116.5:4306 , piece_id=IWXOBJ7G3OY4BYIPPV35S3YH553UXCFSS75Q7KSUVKUHRTC7Z4IA, seg_pos={0 0}, upload=196811]: Done uploading piece. err: ecclient: upload failed (node:1QuFvRW9wGKdKRXPK8UdR8uqTm53CnHaF9CsUz2bUh2Ptw9r1F, address:45.91.116.5:4306): protocol: read tcp 188.166.225.170:48096->45.91.116.5:4306: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:55.096136523 (189h10m49.527216654s) [seg_pos={0 0}, upload=196811]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Aug 16 14:36:55.320533012 (189h10m49.751613143s) [upload=196811]: Done uploading segments... Aug 16 14:36:55.320556706 (189h10m49.751636827s) [upload=196811]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 1m0.226020187s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.226923106s FAIL (2025-08-16 14:36:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== ={0 0}, upload=196820]: Begin upload segment... Aug 16 14:36:23.525955069 (189h10m17.9570352s) [seg_pos={0 0}, upload=196820]: Done begin upload segment. Aug 16 14:36:23.525977080 (189h10m17.957057211s) [noise=true , address=77.90.31.16:4735 , piece_id=VUHDEB62DCUJALFBDSEUJSMIDHVPBITTRH3KJXS3A5XKTOY6BLQA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526216258 (189h10m17.957296389s) [noise=true , address=104.36.82.6:4874 , piece_id=QKXE2NNFS3OWFZXXG6VVZZDXK7ZY5RVJNFLS6XWOEDOVI3TVRRFA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526257237 (189h10m17.957337368s) [noise=true , address=77.90.31.21:4877 , piece_id=764NSONAMBNPL4U7RT5325FGPOWCD2XU4HIEIFAGFZBG3PTHSYOQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526305268 (189h10m17.957385399s) [noise=true , address=77.90.31.13:4627 , piece_id=PYGI2ORIFBAH4AYT7G2FJ3DSKSSCAABDUGBVCUQ6GQEMGSM3LG6Q, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526338433 (189h10m17.957418564s) [noise=true , address=45.91.116.6:4975 , piece_id=KJXDM2YFZAHWYDTLMKH4ZLMZHAUFXGQNJO7DQTX2ES2FUPNB6BVQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526383518 (189h10m17.957463639s) [noise=true , address=104.36.82.12:4151 , piece_id=JV4UYRZ4GM2HTYXG3WUI7UKM2W3AE7ZZ4FNSMXCAIDRUG5VQDC4A, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526413185 (189h10m17.957493316s) [noise=true , address=104.36.82.2:4730 , piece_id=MEPZAHPWMBLUG42GED2DGG6FDPTM2KC5DYKT5LLYA2N7YEOXSNCQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526443663 (189h10m17.957523794s) [noise=true , address=104.36.82.13:4187 , piece_id=KTC5LAVKKYSIRRAANJQN24H5273HCGGXXCZ2K2PCH7BUZ2ERKUUA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526488489 (189h10m17.95756862s) [noise=true , address=77.90.31.21:4878 , piece_id=RKPYQF566SUY6D47M7STHDISLSWAFDRSOQBK7H322CKLALASJSHA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526519969 (189h10m17.9576001s) [noise=true , address=104.36.82.15:4257 , piece_id=RSUG5KCOQ2B2VQPRH6GUQ5PSKY2VD2BJNQGDFQNXW2QXSJUZJZUA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526562952 (189h10m17.957643083s) [noise=true , address=104.36.82.4:4802 , piece_id=5GR33S3YX5YUORL5RJXELMNE4DXAKZ6DFTCRHWTF4UFF2GBUXHZQ, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526596105 (189h10m17.957676236s) [noise=true , address=45.91.116.12:4227 , piece_id=3L5DYJSBR56VXW5OJ6CYLY6NSFOMZUGR2FLNTDUBHTFMMWN6B7MA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526685316 (189h10m17.957765447s) [noise=true , address=104.36.82.3:4767 , piece_id=QRMHSTW2PZK5NFTGL7ZEH2EGOQXWUICL52ISHPDF75FNKMVWLXGA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.526733959 (189h10m17.95781409s) [noise=true , address=45.91.116.4:4938 , piece_id=2CC7MY3GNPTTCNKFYSRMMTQO3Z3RYGVG26ADTZH4KTDKZWATR76A, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.527058720 (189h10m17.958138852s) [noise=true , address=104.36.82.8:4972 , piece_id=N7UO7XL6W7XPEEGPXYWJ6DPMJ22WYCIBJCQJTOHX4CUDIG3B7JAA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:23.527520644 (189h10m17.958600776s) [noise=true , address=104.36.82.11:4115 , piece_id=CZPBOG3IUHUIWTNNYPNZTYHEINRK2RHYMTXL72DM3M3ZIEBUPVNA, seg_pos={0 0}, upload=196820]: Uploading piece... Aug 16 14:36:25.323993740 (189h10m19.755073871s) [upload=196820]: Next returned nil segment Aug 16 14:36:25.324010502 (189h10m19.755090633s) [upload=196820]: Waiting for error group managing segments... Aug 16 14:36:25.695412579 (189h10m20.12649271s) [noise=true , address=104.36.82.11:4115 , piece_id=CZPBOG3IUHUIWTNNYPNZTYHEINRK2RHYMTXL72DM3M3ZIEBUPVNA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.796201608 (189h10m20.227281729s) [noise=true , address=104.36.82.6:4874 , piece_id=QKXE2NNFS3OWFZXXG6VVZZDXK7ZY5RVJNFLS6XWOEDOVI3TVRRFA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.812935036 (189h10m20.244015167s) [noise=true , address=104.36.82.3:4767 , piece_id=QRMHSTW2PZK5NFTGL7ZEH2EGOQXWUICL52ISHPDF75FNKMVWLXGA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:25.951639892 (189h10m20.382720043s) [noise=true , address=104.36.82.2:4730 , piece_id=MEPZAHPWMBLUG42GED2DGG6FDPTM2KC5DYKT5LLYA2N7YEOXSNCQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:26.325196452 (189h10m20.756276583s) [noise=true , address=104.36.82.4:4802 , piece_id=5GR33S3YX5YUORL5RJXELMNE4DXAKZ6DFTCRHWTF4UFF2GBUXHZQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:26.455717299 (189h10m20.88679743s) [noise=true , address=45.91.116.12:4227 , piece_id=3L5DYJSBR56VXW5OJ6CYLY6NSFOMZUGR2FLNTDUBHTFMMWN6B7MA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:29.846714183 (189h10m24.277794314s) [noise=true , address=77.90.31.21:4877 , piece_id=764NSONAMBNPL4U7RT5325FGPOWCD2XU4HIEIFAGFZBG3PTHSYOQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: <nil> Aug 16 14:36:52.425338413 (189h10m46.856418544s) [noise=true , address=77.90.31.16:4735 , piece_id=VUHDEB62DCUJALFBDSEUJSMIDHVPBITTRH3KJXS3A5XKTOY6BLQA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425385362 (189h10m46.856465493s) [noise=true , address=77.90.31.13:4627 , piece_id=PYGI2ORIFBAH4AYT7G2FJ3DSKSSCAABDUGBVCUQ6GQEMGSM3LG6Q, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:45772->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425425078 (189h10m46.856505209s) [noise=true , address=104.36.82.13:4187 , piece_id=KTC5LAVKKYSIRRAANJQN24H5273HCGGXXCZ2K2PCH7BUZ2ERKUUA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: read tcp 188.166.225.170:49434->104.36.82.13:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425450026 (189h10m46.856530157s) [noise=true , address=104.36.82.12:4151 , piece_id=JV4UYRZ4GM2HTYXG3WUI7UKM2W3AE7ZZ4FNSMXCAIDRUG5VQDC4A, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection; write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55154->104.36.82.12:4151: use of closed network connection Aug 16 14:36:52.425457029 (189h10m46.85653716s) [noise=true , address=104.36.82.8:4972 , piece_id=N7UO7XL6W7XPEEGPXYWJ6DPMJ22WYCIBJCQJTOHX4CUDIG3B7JAA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: read tcp 188.166.225.170:43314->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425480404 (189h10m46.856560525s) [noise=true , address=45.91.116.4:4938 , piece_id=2CC7MY3GNPTTCNKFYSRMMTQO3Z3RYGVG26ADTZH4KTDKZWATR76A, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425511253 (189h10m46.856591374s) [noise=true , address=104.36.82.15:4257 , piece_id=RSUG5KCOQ2B2VQPRH6GUQ5PSKY2VD2BJNQGDFQNXW2QXSJUZJZUA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: read tcp 188.166.225.170:47184->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425531041 (189h10m46.856611172s) [noise=true , address=45.91.116.6:4975 , piece_id=KJXDM2YFZAHWYDTLMKH4ZLMZHAUFXGQNJO7DQTX2ES2FUPNB6BVQ, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 188.166.225.170:56156->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425546661 (189h10m46.856626792s) [noise=true , address=77.90.31.21:4878 , piece_id=RKPYQF566SUY6D47M7STHDISLSWAFDRSOQBK7H322CKLALASJSHA, seg_pos={0 0}, upload=196820]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: read tcp 188.166.225.170:58454->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:52.425566118 (189h10m46.856646249s) [seg_pos={0 0}, upload=196820]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 16 14:36:52.651165147 (189h10m47.082245278s) [upload=196820]: Done uploading segments... Aug 16 14:36:52.651195295 (189h10m47.082275426s) [upload=196820]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.226935099s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 16.269432991s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-16 14:36:51 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 16.269445243s edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X...............X............................] duration: 9.109315304s FAIL (2025-08-16 14:36:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 16 14:15:15.081221025 (188h49m9.512301156s) [upload=196448]: Starting upload Aug 16 14:15:15.081268716 (188h49m9.512348847s) [upload=196448]: Done starting upload Aug 16 14:15:15.081327208 (188h49m9.512407339s) [upload=196448]: Uploading segments... Aug 16 14:15:15.081365671 (188h49m9.512445802s) [upload=196448]: Got next segment. Inline: true Aug 16 14:15:15.081369339 (188h49m9.51244946s) [upload=196448]: Waiting for error group managing segments... Aug 16 14:15:15.081400809 (188h49m9.51248094s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:15:15.081404686 (188h49m9.512484817s) [upload=196448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:15:15.081407071 (188h49m9.512487202s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:15:16.556926721 (188h49m10.988006852s) [upload=196448]: Done uploading segments... Aug 16 14:15:16.556955035 (188h49m10.988035166s) [upload=196448]: Upload finished. err: <nil> Duration:* 9.109338989s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000889431s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-16 14:36:35 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 30.000942142s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000489256s FAIL (2025-08-16 14:36:33 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 30.000516327s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................X........XXX] duration: 30.00032931s FAIL (2025-08-16 14:36:31 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 30.000348175s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.235694412s FAIL (2025-08-16 14:36:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== upload=196809]: Uploading piece... Aug 16 14:35:52.850987545 (189h9m47.282067736s) [noise=true , address=45.91.116.10:4083 , piece_id=QUC25GCSV7DCBRHH3UE27Z6LAIYG2ECIGBQ5IYEM54L6U25E4XUQ, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.851719235 (189h9m47.282799366s) [noise=true , address=104.36.82.15:4258 , piece_id=FTVUGPY7TD3N6IDTADSP7QGRF7VCUV6TA6OIDSGZ3NBZBLEI5QPQ, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.852079835 (189h9m47.283159966s) [noise=true , address=104.36.82.7:4912 , piece_id=O2NBBCGYGFTNFUK527VXS4XGDW75TFYVEHNTR6AD6C3WKPXD2JJQ, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.852252435 (189h9m47.283332566s) [noise=true , address=104.36.82.12:4149 , piece_id=UQ7XXFVRW6NAUYRAEFCCNWFWPDGCSBCZ5T6QZBEOTQYRAZ64ULNQ, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.852868184 (189h9m47.283948315s) [noise=true , address=77.90.31.15:4697 , piece_id=44WVIPCO6EVKFSGJZILYJYTQ2Z42AYQBWJQJWDX4CZA6WKV7UMWA, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.853115808 (189h9m47.284195939s) [noise=true , address=104.36.82.11:4114 , piece_id=3PBCLVTLAHF5UVZ5ASEW7EBNXLKESEM6Y3BT7DRNMFFXXDVISEXA, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.853502888 (189h9m47.284583019s) [noise=true , address=104.36.82.15:4259 , piece_id=SZB3QJIZSGMAGRW43ROMFPGZK55HFFIIU6KFPYRFC6MAPABMNEFQ, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.853732156 (189h9m47.284812287s) [noise=true , address=45.91.116.5:4299 , piece_id=SXQUYA6JMY4EE5HPAO3EB6KHFZUTCYGTHIX77ZA4H5ETUFSZH34A, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.853748218 (189h9m47.284828349s) [noise=true , address=104.36.82.13:4187 , piece_id=36DIEIDHVA5NXCMW3X7WMV2J7GSXKEQLPA24F3FNYO4MGWGVEYYQ, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.853830214 (189h9m47.284910345s) [noise=true , address=104.36.82.13:4185 , piece_id=P3BSX4DRLI3TWVAFOXQHLKWWLRPAPRBQ7VFYOUWNRDCUHQTJACMA, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.854291868 (189h9m47.285371999s) [noise=true , address=77.90.31.10:4520 , piece_id=ADFY7R3747NF7IO6X6ZIDGHV4VDUSB5ZKEPEFHETXLG3MYO7PW4Q, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.854306085 (189h9m47.285386216s) [noise=true , address=104.36.82.6:4876 , piece_id=5V6YMFIWNI7D6SMOTM3TA4ZZVHKLERBINMJXTSPVM3RJV5B7XWAQ, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.855212990 (189h9m47.286293121s) [noise=true , address=77.90.31.18:4807 , piece_id=W7RTOXPUUTTNDAMHRAPUIIOXQ4LQ7RCHEL62AUMYP453QGBNU52Q, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.855473809 (189h9m47.28655394s) [noise=true , address=104.36.82.14:4222 , piece_id=JH4NYKGHUXTWLSC7Y6EWS3PU5DNEMHKMEZ3AXBKJJ6YSQJCN54CA, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:52.855713898 (189h9m47.286794029s) [noise=true , address=104.36.82.8:4970 , piece_id=MEQUZ4BN6C2UXIIK7I6ICY5DUQW3GAI4IEVDQLDX7QI75SGMEUXA, seg_pos={0 0}, upload=196809]: Uploading piece... Aug 16 14:35:54.784505499 (189h9m49.21558563s) [upload=196809]: Next returned nil segment Aug 16 14:35:54.784522421 (189h9m49.215602552s) [upload=196809]: Waiting for error group managing segments... Aug 16 14:35:55.102232154 (189h9m49.533312285s) [noise=true , address=104.36.82.15:4258 , piece_id=FTVUGPY7TD3N6IDTADSP7QGRF7VCUV6TA6OIDSGZ3NBZBLEI5QPQ, seg_pos={0 0}, upload=196809]: Done uploading piece. err: <nil> Aug 16 14:35:55.965554157 (189h9m50.396634288s) [noise=true , address=77.90.31.15:4697 , piece_id=44WVIPCO6EVKFSGJZILYJYTQ2Z42AYQBWJQJWDX4CZA6WKV7UMWA, seg_pos={0 0}, upload=196809]: Done uploading piece. err: <nil> Aug 16 14:35:56.078423109 (189h9m50.50950324s) [noise=true , address=45.91.116.10:4083 , piece_id=QUC25GCSV7DCBRHH3UE27Z6LAIYG2ECIGBQ5IYEM54L6U25E4XUQ, seg_pos={0 0}, upload=196809]: Done uploading piece. err: <nil> Aug 16 14:36:05.199777165 (189h9m59.630857296s) [noise=true , address=104.36.82.9:4006 , piece_id=EPBVDB6EONUDQCIKNGEU6SVGKBCWY4PFZEFEBI62R2M46QE3XVBA, seg_pos={0 0}, upload=196809]: Done uploading piece. err: <nil> Aug 16 14:36:13.048108341 (189h10m7.479188472s) [noise=true , address=104.36.82.7:4912 , piece_id=O2NBBCGYGFTNFUK527VXS4XGDW75TFYVEHNTR6AD6C3WKPXD2JJQ, seg_pos={0 0}, upload=196809]: Done uploading piece. err: <nil> Aug 16 14:36:21.965404685 (189h10m16.396484816s) [noise=true , address=104.36.82.13:4185 , piece_id=P3BSX4DRLI3TWVAFOXQHLKWWLRPAPRBQ7VFYOUWNRDCUHQTJACMA, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: rpc: dial tcp 104.36.82.13:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965450924 (189h10m16.396531055s) [noise=true , address=104.36.82.12:4149 , piece_id=UQ7XXFVRW6NAUYRAEFCCNWFWPDGCSBCZ5T6QZBEOTQYRAZ64ULNQ, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965677016 (189h10m16.396757148s) [noise=true , address=104.36.82.11:4114 , piece_id=3PBCLVTLAHF5UVZ5ASEW7EBNXLKESEM6Y3BT7DRNMFFXXDVISEXA, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: read tcp 188.166.225.170:47022->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965685663 (189h10m16.396765794s) [noise=true , address=77.90.31.18:4807 , piece_id=W7RTOXPUUTTNDAMHRAPUIIOXQ4LQ7RCHEL62AUMYP453QGBNU52Q, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: rpc: dial tcp 77.90.31.18:4807: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965735098 (189h10m16.396815219s) [noise=true , address=77.90.31.10:4520 , piece_id=ADFY7R3747NF7IO6X6ZIDGHV4VDUSB5ZKEPEFHETXLG3MYO7PW4Q, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 188.166.225.170:58072->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965777820 (189h10m16.396857951s) [noise=true , address=104.36.82.6:4876 , piece_id=5V6YMFIWNI7D6SMOTM3TA4ZZVHKLERBINMJXTSPVM3RJV5B7XWAQ, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: read tcp 188.166.225.170:43870->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965792377 (189h10m16.396872508s) [noise=true , address=104.36.82.13:4187 , piece_id=36DIEIDHVA5NXCMW3X7WMV2J7GSXKEQLPA24F3FNYO4MGWGVEYYQ, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: read tcp 188.166.225.170:36838->104.36.82.13:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965833396 (189h10m16.396913527s) [noise=true , address=104.36.82.15:4259 , piece_id=SZB3QJIZSGMAGRW43ROMFPGZK55HFFIIU6KFPYRFC6MAPABMNEFQ, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: read tcp 188.166.225.170:47394->104.36.82.15:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965868914 (189h10m16.396949045s) [noise=true , address=45.91.116.5:4299 , piece_id=SXQUYA6JMY4EE5HPAO3EB6KHFZUTCYGTHIX77ZA4H5ETUFSZH34A, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 188.166.225.170:45142->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965903189 (189h10m16.39698332s) [noise=true , address=104.36.82.14:4222 , piece_id=JH4NYKGHUXTWLSC7Y6EWS3PU5DNEMHKMEZ3AXBKJJ6YSQJCN54CA, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: read tcp 188.166.225.170:47514->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965932735 (189h10m16.397012867s) [noise=true , address=104.36.82.8:4970 , piece_id=MEQUZ4BN6C2UXIIK7I6ICY5DUQW3GAI4IEVDQLDX7QI75SGMEUXA, seg_pos={0 0}, upload=196809]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: read tcp 188.166.225.170:42902->104.36.82.8:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:36:21.965981068 (189h10m16.397061199s) [seg_pos={0 0}, upload=196809]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Aug 16 14:36:22.199291292 (189h10m16.630371413s) [upload=196809]: Done uploading segments... Aug 16 14:36:22.199317603 (189h10m16.630397734s) [upload=196809]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.235722224s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.......X...X...........X................................X.] duration: 12.880438613s FAIL (2025-08-16 14:35:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 16 14:34:40.263571488 (189h8m34.694651619s) [upload=196789]: Starting upload Aug 16 14:34:40.263631312 (189h8m34.694711443s) [upload=196789]: Done starting upload Aug 16 14:34:40.263714171 (189h8m34.694794302s) [upload=196789]: Uploading segments... Aug 16 14:34:40.263813581 (189h8m34.694893712s) [upload=196789]: Got next segment. Inline: false Aug 16 14:34:40.263853217 (189h8m34.694933348s) [upload=196789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:34:40.263857726 (189h8m34.694937857s) [upload=196789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 14:34:40.833063349 (189h8m35.26414347s) [seg_pos={0 0}, upload=196789]: Begin upload segment... Aug 16 14:34:40.833110649 (189h8m35.26419078s) [seg_pos={0 0}, upload=196789]: Done begin upload segment. Aug 16 14:34:40.833132070 (189h8m35.264212201s) [noise=true , address=45.91.116.14:4198 , piece_id=3QDHIGMC4BVOBCLR3NMSRGNXKB6T354BZBBXW7D6XGUIP5UYAEMA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.833251768 (189h8m35.26433194s) [noise=true , address=104.36.82.44:4889 , piece_id=S2DNNAXK5YHKHBENEO2OD3JEYSILNFLEUCXH5O4ZBPZGWFCKRMMQ, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.833520683 (189h8m35.264600804s) [noise=true , address=104.36.82.23:4474 , piece_id=Y3YM3UEB2DOAGZGX6IDEGWQP4PQPQWRFZOYQWHAIFNAAXKOMVSQQ, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.833769830 (189h8m35.264849961s) [noise=true , address=104.36.82.5:4871 , piece_id=TMSGJVUUSKPAZRPIIB5NLBMKWLS6X7YFGBOEE5N2M7TSRRLFI7ZA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.833831448 (189h8m35.26491158s) [noise=true , address=77.90.31.15:4725 , piece_id=TD652IP6PITO3S7CGNEHIARRUQKV62CI4TCSVXVDBHZT33LMXUFA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834118728 (189h8m35.265198859s) [noise=true , address=104.36.82.4:4830 , piece_id=OVM2QFYFNGEJQEIUWDORN5K4TIEQ5AL3UIRPEIJF6XM67T5SUTKA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834144978 (189h8m35.265225109s) [noise=true , address=156.146.43.225:4457 , piece_id=RW6NEMNN7HZ65GAV2FLT333AYJ4VUTLOMU3VL6ZXG6ZFIXPMWZ5A, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834399895 (189h8m35.265480026s) [noise=true , address=104.36.82.38:4704 , piece_id=UWTC6MN2PUIHJDO7LUXXU6VU5E6AWHTFR56J3GVUMKRV3X5WXZRA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834450522 (189h8m35.265530653s) [noise=true , address=104.36.82.59:4310 , piece_id=W2KH577KXKLJDJ46IRDOKWDZPHCYVWVUUCUEKFVC2UN5FYCBVM6Q, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834614957 (189h8m35.265695088s) [noise=true , address=104.36.82.39:4734 , piece_id=NRFIPHK7CDGM33Y7XMO25TJY2Y3RY4V6L556HTKRTIMB2PI2FUJA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834722313 (189h8m35.265802444s) [noise=true , address=77.90.31.14:4690 , piece_id=UUAPEVRI2NDXSM3DIE2B4T2B3I5TRUKVFZS7S6KDOYEXGTUSIUYA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834823196 (189h8m35.265903327s) [noise=true , address=79.127.201.215:4347 , piece_id=HL7OPMF7QJXON7JSQECHAVBNREDXPM64JNMZQ5RHIS7YGSOXXUDA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834996417 (189h8m35.266076548s) [noise=true , address=104.36.82.60:4351 , piece_id=XGDNZF5S2I55CZY6VWAYZCVZAGLDK2S2JPTN7RKYS3ZVIOJP4N3A, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835024831 (189h8m35.266104962s) [noise=true , address=66.23.193.175:4556 , piece_id=PYTADEXG2CU3ORDZHRAUQJGLF35NPYPPQMDWZVFBQ4SIQBEOEHLA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835269409 (189h8m35.26634953s) [noise=true , address=77.90.31.12:4602 , piece_id=TX5WQHHAX5I6F6F7ZDEP7GNB4E5SGFMVICWXQGCXRVNICFT6D7AA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835273848 (189h8m35.266353979s) [noise=true , address=77.90.31.13:4648 , piece_id=AAOHQWQC6BQX6SBBKVSC7VR3GA5UYIRBULI4YK7S5KPD7CQKPOWQ, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835469582 (189h8m35.266549713s) [noise=true , address=45.91.116.13:4178 , piece_id=XLASEEFYZZCK2EHBDRISOXG6S5G6ITDXPDNEUNA5RMGPVFFW4ZEA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835521712 (189h8m35.266601843s) [noise=true , address=77.90.31.19:4857 , piece_id=UO3T4HPP7ELDIBODRDUVZABBFH4E2AQ4DMVTVNRP6QF2ZOTZJBRA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:42.007861727 (189h8m36.438941859s) [upload=196789]: Next returned nil segment Aug 16 14:34:42.007878810 (189h8m36.438958941s) [upload=196789]: Waiting for error group managing segments... Aug 16 14:34:42.315392914 (189h8m36.746473045s) [noise=true , address=66.23.193.175:4556 , piece_id=PYTADEXG2CU3ORDZHRAUQJGLF35NPYPPQMDWZVFBQ4SIQBEOEHLA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:42.776859961 (189h8m37.207940082s) [noise=true , address=156.146.43.225:4457 , piece_id=RW6NEMNN7HZ65GAV2FLT333AYJ4VUTLOMU3VL6ZXG6ZFIXPMWZ5A, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:42.857477395 (189h8m37.288557526s) [noise=true , address=79.127.201.215:4347 , piece_id=HL7OPMF7QJXON7JSQECHAVBNREDXPM64JNMZQ5RHIS7YGSOXXUDA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:42.861840152 (189h8m37.292920283s) [noise=true , address=104.36.82.59:4310 , piece_id=W2KH577KXKLJDJ46IRDOKWDZPHCYVWVUUCUEKFVC2UN5FYCBVM6Q, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:42.930475047 (189h8m37.361555168s) [noise=true , address=104.36.82.5:4871 , piece_id=TMSGJVUUSKPAZRPIIB5NLBMKWLS6X7YFGBOEE5N2M7TSRRLFI7ZA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:45.511139098 (189h8m39.942219229s) [noise=true , address=104.36.82.60:4351 , piece_id=XGDNZF5S2I55CZY6VWAYZCVZAGLDK2S2JPTN7RKYS3ZVIOJP4N3A, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:46.282920207 (189h8m40.714000338s) [noise=true , address=104.36.82.4:4830 , piece_id=OVM2QFYFNGEJQEIUWDORN5K4TIEQ5AL3UIRPEIJF6XM67T5SUTKA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:50.089234608 (189h8m44.520314739s) [noise=true , address=104.36.82.44:4889 , piece_id=S2DNNAXK5YHKHBENEO2OD3JEYSILNFLEUCXH5O4ZBPZGWFCKRMMQ, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:50.456858006 (189h8m44.887938157s) [noise=true , address=45.91.116.14:4198 , piece_id=3QDHIGMC4BVOBCLR3NMSRGNXKB6T354BZBBXW7D6XGUIP5UYAEMA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:54.178960324 (189h8m48.610040455s) [noise=true , address=104.36.82.39:4734 , piece_id=NRFIPHK7CDGM33Y7XMO25TJY2Y3RY4V6L556HTKRTIMB2PI2FUJA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:35:03.464457833 (189h8m57.895537964s) [noise=true , address=77.90.31.15:4725 , piece_id=TD652IP6PITO3S7CGNEHIARRUQKV62CI4TCSVXVDBHZT33LMXUFA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:35:39.614950309 (189h9m34.04603044s) [noise=true , address=104.36.82.23:4474 , piece_id=Y3YM3UEB2DOAGZGX6IDEGWQP4PQPQWRFZOYQWHAIFNAAXKOMVSQQ, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: read tcp 188.166.225.170:52308->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615000916 (189h9m34.046081047s) [noise=true , address=45.91.116.13:4178 , piece_id=XLASEEFYZZCK2EHBDRISOXG6S5G6ITDXPDNEUNA5RMGPVFFW4ZEA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:125m3sbLAzXdKnjK9WZmasPccheZX63eDagUXqxVCPiakA9ApNX, address:45.91.116.13:4178): protocol: read tcp 188.166.225.170:39408->45.91.116.13:4178: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615034250 (189h9m34.046114381s) [noise=true , address=77.90.31.19:4857 , piece_id=UO3T4HPP7ELDIBODRDUVZABBFH4E2AQ4DMVTVNRP6QF2ZOTZJBRA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:12i57FF97oEddYLZpNxSq5or9WLEquL8ZW1cVUuW4AThBbDqcCL, address:77.90.31.19:4857): protocol: read tcp 188.166.225.170:37302->77.90.31.19:4857: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615089225 (189h9m34.046169356s) [noise=true , address=104.36.82.38:4704 , piece_id=UWTC6MN2PUIHJDO7LUXXU6VU5E6AWHTFR56J3GVUMKRV3X5WXZRA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1225CoSQvGuiWjJZWaXhvvq4gxzMymZqUb9fBrfLWw4RekwAEhp, address:104.36.82.38:4704): protocol: write tcp 188.166.225.170:60564->104.36.82.38:4704: use of closed network connection; write tcp 188.166.225.170:60564->104.36.82.38:4704: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60564->104.36.82.38:4704: use of closed network connection Aug 16 14:35:39.615113432 (189h9m34.046193553s) [noise=true , address=77.90.31.14:4690 , piece_id=UUAPEVRI2NDXSM3DIE2B4T2B3I5TRUKVFZS7S6KDOYEXGTUSIUYA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: read tcp 188.166.225.170:50794->77.90.31.14:4690: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615120976 (189h9m34.046201127s) [noise=true , address=77.90.31.13:4648 , piece_id=AAOHQWQC6BQX6SBBKVSC7VR3GA5UYIRBULI4YK7S5KPD7CQKPOWQ, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1niTsTaHjA6LoMMrkEsSnLcRDhZtVAjxsbXUXVeXzhUp3FPbWh, address:77.90.31.13:4648): protocol: read tcp 188.166.225.170:36442->77.90.31.13:4648: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615171973 (189h9m34.046252125s) [noise=true , address=77.90.31.12:4602 , piece_id=TX5WQHHAX5I6F6F7ZDEP7GNB4E5SGFMVICWXQGCXRVNICFT6D7AA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: read tcp 188.166.225.170:42998->77.90.31.12:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615216529 (189h9m34.04629671s) [seg_pos={0 0}, upload=196789]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 16 14:35:41.235994622 (189h9m35.667074743s) [upload=196789]: Done uploading segments... Aug 16 14:35:41.236021373 (189h9m35.667101504s) [upload=196789]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 12.880453372s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X...............X...........................] duration: 3.732428726s FAIL (2025-08-16 14:35:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 16 14:15:15.081221025 (188h49m9.512301156s) [upload=196448]: Starting upload Aug 16 14:15:15.081268716 (188h49m9.512348847s) [upload=196448]: Done starting upload Aug 16 14:15:15.081327208 (188h49m9.512407339s) [upload=196448]: Uploading segments... Aug 16 14:15:15.081365671 (188h49m9.512445802s) [upload=196448]: Got next segment. Inline: true Aug 16 14:15:15.081369339 (188h49m9.51244946s) [upload=196448]: Waiting for error group managing segments... Aug 16 14:15:15.081400809 (188h49m9.51248094s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:15:15.081404686 (188h49m9.512484817s) [upload=196448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:15:15.081407071 (188h49m9.512487202s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:15:16.556926721 (188h49m10.988006852s) [upload=196448]: Done uploading segments... Aug 16 14:15:16.556955035 (188h49m10.988035166s) [upload=196448]: Upload finished. err: <nil> Duration:* 3.732443304s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 11.631146603s FAIL (2025-08-16 14:35:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== QQ7WTQUAHTJ5TXV75E733HJFA633P5JRA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.951628041 (189h8m29.382708172s) [noise=true , address=66.23.193.175:4563 , piece_id=5GG45NDIMJ7IS6VN5ZXVDEK4AJHXWYM3NYECQ57UL53KI4KZ7DBA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.951836219 (189h8m29.38291635s) [noise=true , address=104.36.82.2:4753 , piece_id=375FY6WK5JXQ2COG6IGBKC7PSM2ANELMJM6ZTXUHSDBSXGRUKGQQ, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952052283 (189h8m29.383132414s) [noise=true , address=104.36.82.44:4882 , piece_id=PKPB7J6QD4W454FMSJGARPXEPVFDASVWMS5443OZ7KIJIIVEVYVA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952254570 (189h8m29.383334701s) [noise=true , address=104.36.82.3:4769 , piece_id=J3AJIDJZJLCBKG2GCTINZBJIN32Y6KOZWQTOTP6DQKOLX6ERLE6A, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952498887 (189h8m29.383579008s) [noise=true , address=77.90.31.13:4655 , piece_id=VRYJ2VRUAHTINO53SJI774HW47S3OEGUGSQIY57D34BESSUTKNBQ, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952698509 (189h8m29.38377864s) [noise=true , address=77.90.31.16:4764 , piece_id=DJPW6AZMK4P43AL4UZG7MP6UCXCK2K72TS33VD4JYNG4WH535HLA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952902660 (189h8m29.383982781s) [noise=true , address=77.90.31.12:4619 , piece_id=L3RZOAULMD4AVOG2NIB3RU44IUPCXIURMPQJCRZBIVNJN2YVIDNA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.953096290 (189h8m29.384176422s) [noise=true , address=77.90.31.11:4585 , piece_id=MLXWH6KZS6ZDST4DZFI3QC652LWUNQNJ5SMS32UQS65TDWYILBHA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:36.088091971 (189h8m30.519172102s) [upload=196785]: Next returned nil segment Aug 16 14:34:36.088130115 (189h8m30.519210246s) [upload=196785]: Waiting for error group managing segments... Aug 16 14:34:36.405455542 (189h8m30.836535673s) [noise=true , address=66.23.193.175:4563 , piece_id=5GG45NDIMJ7IS6VN5ZXVDEK4AJHXWYM3NYECQ57UL53KI4KZ7DBA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:36.792259310 (189h8m31.223339431s) [noise=true , address=79.127.220.101:4073 , piece_id=H6IU6XQGKLLNEU7WSYKY2XA5G7VMOUH6KZ65BIQIQLUKBVJUPLHA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:37.146333052 (189h8m31.577413173s) [noise=true , address=79.127.201.215:4346 , piece_id=DA7IVKEOBVCOKUHBUMJY2M2IS2IUE7WK34MATIN6WTET5P5R5PVA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:38.678171978 (189h8m33.109252099s) [noise=true , address=77.90.31.13:4655 , piece_id=VRYJ2VRUAHTINO53SJI774HW47S3OEGUGSQIY57D34BESSUTKNBQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:55.651073853 (189h8m50.082153984s) [noise=true , address=104.36.82.38:4712 , piece_id=4JQB47DETHX7OY3W5HOKTWMAF6Q577ZFWOTHGPMLQOS5MTPFGO6A, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:57.241163206 (189h8m51.672243408s) [noise=true , address=104.36.82.59:4309 , piece_id=LPJ3RMEXVALSGSBWWGLSSGWY4YOAUNXPT2QGSHXIRBPG66BVUCXQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:35:33.848810590 (189h9m28.279890711s) [noise=true , address=77.90.31.18:4834 , piece_id=XQ3TGTB3Q2W6QRL7A5SWQ3CTYFB46CXARY6IOAWGSXB4QAB5N64Q, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:50154->77.90.31.18:4834: use of closed network connection; write tcp 188.166.225.170:50154->77.90.31.18:4834: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50154->77.90.31.18:4834: use of closed network connection Aug 16 14:35:33.848868651 (189h9m28.279948772s) [noise=true , address=104.36.82.44:4882 , piece_id=PKPB7J6QD4W454FMSJGARPXEPVFDASVWMS5443OZ7KIJIIVEVYVA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:104.36.82.44:4882): protocol: write tcp 188.166.225.170:53334->104.36.82.44:4882: use of closed network connection; write tcp 188.166.225.170:53334->104.36.82.44:4882: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53334->104.36.82.44:4882: use of closed network connection Aug 16 14:35:33.848905823 (189h9m28.279985954s) [noise=true , address=104.36.82.3:4769 , piece_id=J3AJIDJZJLCBKG2GCTINZBJIN32Y6KOZWQTOTP6DQKOLX6ERLE6A, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:104.36.82.3:4769): protocol: read tcp 188.166.225.170:43434->104.36.82.3:4769: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.848933035 (189h9m28.280013166s) [noise=true , address=77.90.31.12:4619 , piece_id=L3RZOAULMD4AVOG2NIB3RU44IUPCXIURMPQJCRZBIVNJN2YVIDNA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX, address:77.90.31.12:4619): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 14:35:33.848947472 (189h9m28.280027603s) [noise=true , address=104.36.82.60:4365 , piece_id=7XG5RETE5D4XCRNDTEZSDEGXJ4PKUEXRH65F4I2LUSLORWN4JOUQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: read tcp 188.166.225.170:33404->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849021173 (189h9m28.280101304s) [noise=true , address=77.90.31.16:4764 , piece_id=DJPW6AZMK4P43AL4UZG7MP6UCXCK2K72TS33VD4JYNG4WH535HLA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT, address:77.90.31.16:4764): protocol: write tcp 188.166.225.170:49000->77.90.31.16:4764: use of closed network connection; write tcp 188.166.225.170:49000->77.90.31.16:4764: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49000->77.90.31.16:4764: use of closed network connection Aug 16 14:35:33.849117999 (189h9m28.28019813s) [noise=true , address=77.90.31.11:4585 , piece_id=MLXWH6KZS6ZDST4DZFI3QC652LWUNQNJ5SMS32UQS65TDWYILBHA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1Tve8WUWhJCK2cVWLbxxnz3WVzR2bi2wLx5XrF6QbhSN4HcsQG, address:77.90.31.11:4585): protocol: write tcp 188.166.225.170:56128->77.90.31.11:4585: use of closed network connection; write tcp 188.166.225.170:56128->77.90.31.11:4585: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56128->77.90.31.11:4585: use of closed network connection Aug 16 14:35:33.849161221 (189h9m28.280241342s) [noise=true , address=77.90.31.19:4875 , piece_id=7LF3JKR5YQO2IAKOCC6QQ7WTQUAHTJ5TXV75E733HJFA633P5JRA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1xSFQaW3aVnAVDYE5r5ADmCGU95vWnshuZyte32W2YAj94avNn, address:77.90.31.19:4875): protocol: write tcp 188.166.225.170:38064->77.90.31.19:4875: use of closed network connection; write tcp 188.166.225.170:38064->77.90.31.19:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38064->77.90.31.19:4875: use of closed network connection Aug 16 14:35:33.849200948 (189h9m28.280281079s) [noise=true , address=104.36.82.5:4843 , piece_id=JZOHQO4YX7227CIM6TGAQ7QJ4JJIUVE2YC5RZ6UIDTYMOESTWMAQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: read tcp 188.166.225.170:35936->104.36.82.5:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849300387 (189h9m28.280380518s) [noise=true , address=104.36.82.2:4753 , piece_id=375FY6WK5JXQ2COG6IGBKC7PSM2ANELMJM6ZTXUHSDBSXGRUKGQQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:104.36.82.2:4753): protocol: read tcp 188.166.225.170:45972->104.36.82.2:4753: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849333711 (189h9m28.280413842s) [noise=true , address=104.36.82.39:4760 , piece_id=IQ3MTMPLF4TUASFFY3U2O325XK5VZG7MK3P2HLEKOSBKFVV64S5Q, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1jJcppBK9zJzGedgY9KEjEA8twqtkdZcjhcnXJoN5AWi5QzR4E, address:104.36.82.39:4760): protocol: read tcp 188.166.225.170:41552->104.36.82.39:4760: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849343871 (189h9m28.280424002s) [noise=true , address=104.36.82.18:4389 , piece_id=YMIA2XHWMZZRHULDAP7VLTMU5COOQOMAYLSCAWZ2RVH6FAHGZNVA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1mMqCXPNzFx4cnu8Nb3gNKcJFJ4tGJjnkwY2EVPjUBnwNVcM9R, address:104.36.82.18:4389): protocol: read tcp 188.166.225.170:60738->104.36.82.18:4389: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849392264 (189h9m28.280472395s) [seg_pos={0 0}, upload=196785]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 16 14:35:34.082190547 (189h9m28.513270678s) [upload=196785]: Done uploading segments... Aug 16 14:35:34.082221266 (189h9m28.513301397s) [upload=196785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 11.631167063s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.......X...X...........X................................X] duration: 1m1.62301992s FAIL (2025-08-16 14:35:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== Aug 16 14:34:40.263571488 (189h8m34.694651619s) [upload=196789]: Starting upload Aug 16 14:34:40.263631312 (189h8m34.694711443s) [upload=196789]: Done starting upload Aug 16 14:34:40.263714171 (189h8m34.694794302s) [upload=196789]: Uploading segments... Aug 16 14:34:40.263813581 (189h8m34.694893712s) [upload=196789]: Got next segment. Inline: false Aug 16 14:34:40.263853217 (189h8m34.694933348s) [upload=196789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:34:40.263857726 (189h8m34.694937857s) [upload=196789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 14:34:40.833063349 (189h8m35.26414347s) [seg_pos={0 0}, upload=196789]: Begin upload segment... Aug 16 14:34:40.833110649 (189h8m35.26419078s) [seg_pos={0 0}, upload=196789]: Done begin upload segment. Aug 16 14:34:40.833132070 (189h8m35.264212201s) [noise=true , address=45.91.116.14:4198 , piece_id=3QDHIGMC4BVOBCLR3NMSRGNXKB6T354BZBBXW7D6XGUIP5UYAEMA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.833251768 (189h8m35.26433194s) [noise=true , address=104.36.82.44:4889 , piece_id=S2DNNAXK5YHKHBENEO2OD3JEYSILNFLEUCXH5O4ZBPZGWFCKRMMQ, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.833520683 (189h8m35.264600804s) [noise=true , address=104.36.82.23:4474 , piece_id=Y3YM3UEB2DOAGZGX6IDEGWQP4PQPQWRFZOYQWHAIFNAAXKOMVSQQ, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.833769830 (189h8m35.264849961s) [noise=true , address=104.36.82.5:4871 , piece_id=TMSGJVUUSKPAZRPIIB5NLBMKWLS6X7YFGBOEE5N2M7TSRRLFI7ZA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.833831448 (189h8m35.26491158s) [noise=true , address=77.90.31.15:4725 , piece_id=TD652IP6PITO3S7CGNEHIARRUQKV62CI4TCSVXVDBHZT33LMXUFA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834118728 (189h8m35.265198859s) [noise=true , address=104.36.82.4:4830 , piece_id=OVM2QFYFNGEJQEIUWDORN5K4TIEQ5AL3UIRPEIJF6XM67T5SUTKA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834144978 (189h8m35.265225109s) [noise=true , address=156.146.43.225:4457 , piece_id=RW6NEMNN7HZ65GAV2FLT333AYJ4VUTLOMU3VL6ZXG6ZFIXPMWZ5A, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834399895 (189h8m35.265480026s) [noise=true , address=104.36.82.38:4704 , piece_id=UWTC6MN2PUIHJDO7LUXXU6VU5E6AWHTFR56J3GVUMKRV3X5WXZRA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834450522 (189h8m35.265530653s) [noise=true , address=104.36.82.59:4310 , piece_id=W2KH577KXKLJDJ46IRDOKWDZPHCYVWVUUCUEKFVC2UN5FYCBVM6Q, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834614957 (189h8m35.265695088s) [noise=true , address=104.36.82.39:4734 , piece_id=NRFIPHK7CDGM33Y7XMO25TJY2Y3RY4V6L556HTKRTIMB2PI2FUJA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834722313 (189h8m35.265802444s) [noise=true , address=77.90.31.14:4690 , piece_id=UUAPEVRI2NDXSM3DIE2B4T2B3I5TRUKVFZS7S6KDOYEXGTUSIUYA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834823196 (189h8m35.265903327s) [noise=true , address=79.127.201.215:4347 , piece_id=HL7OPMF7QJXON7JSQECHAVBNREDXPM64JNMZQ5RHIS7YGSOXXUDA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.834996417 (189h8m35.266076548s) [noise=true , address=104.36.82.60:4351 , piece_id=XGDNZF5S2I55CZY6VWAYZCVZAGLDK2S2JPTN7RKYS3ZVIOJP4N3A, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835024831 (189h8m35.266104962s) [noise=true , address=66.23.193.175:4556 , piece_id=PYTADEXG2CU3ORDZHRAUQJGLF35NPYPPQMDWZVFBQ4SIQBEOEHLA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835269409 (189h8m35.26634953s) [noise=true , address=77.90.31.12:4602 , piece_id=TX5WQHHAX5I6F6F7ZDEP7GNB4E5SGFMVICWXQGCXRVNICFT6D7AA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835273848 (189h8m35.266353979s) [noise=true , address=77.90.31.13:4648 , piece_id=AAOHQWQC6BQX6SBBKVSC7VR3GA5UYIRBULI4YK7S5KPD7CQKPOWQ, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835469582 (189h8m35.266549713s) [noise=true , address=45.91.116.13:4178 , piece_id=XLASEEFYZZCK2EHBDRISOXG6S5G6ITDXPDNEUNA5RMGPVFFW4ZEA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:40.835521712 (189h8m35.266601843s) [noise=true , address=77.90.31.19:4857 , piece_id=UO3T4HPP7ELDIBODRDUVZABBFH4E2AQ4DMVTVNRP6QF2ZOTZJBRA, seg_pos={0 0}, upload=196789]: Uploading piece... Aug 16 14:34:42.007861727 (189h8m36.438941859s) [upload=196789]: Next returned nil segment Aug 16 14:34:42.007878810 (189h8m36.438958941s) [upload=196789]: Waiting for error group managing segments... Aug 16 14:34:42.315392914 (189h8m36.746473045s) [noise=true , address=66.23.193.175:4556 , piece_id=PYTADEXG2CU3ORDZHRAUQJGLF35NPYPPQMDWZVFBQ4SIQBEOEHLA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:42.776859961 (189h8m37.207940082s) [noise=true , address=156.146.43.225:4457 , piece_id=RW6NEMNN7HZ65GAV2FLT333AYJ4VUTLOMU3VL6ZXG6ZFIXPMWZ5A, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:42.857477395 (189h8m37.288557526s) [noise=true , address=79.127.201.215:4347 , piece_id=HL7OPMF7QJXON7JSQECHAVBNREDXPM64JNMZQ5RHIS7YGSOXXUDA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:42.861840152 (189h8m37.292920283s) [noise=true , address=104.36.82.59:4310 , piece_id=W2KH577KXKLJDJ46IRDOKWDZPHCYVWVUUCUEKFVC2UN5FYCBVM6Q, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:42.930475047 (189h8m37.361555168s) [noise=true , address=104.36.82.5:4871 , piece_id=TMSGJVUUSKPAZRPIIB5NLBMKWLS6X7YFGBOEE5N2M7TSRRLFI7ZA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:45.511139098 (189h8m39.942219229s) [noise=true , address=104.36.82.60:4351 , piece_id=XGDNZF5S2I55CZY6VWAYZCVZAGLDK2S2JPTN7RKYS3ZVIOJP4N3A, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:46.282920207 (189h8m40.714000338s) [noise=true , address=104.36.82.4:4830 , piece_id=OVM2QFYFNGEJQEIUWDORN5K4TIEQ5AL3UIRPEIJF6XM67T5SUTKA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:50.089234608 (189h8m44.520314739s) [noise=true , address=104.36.82.44:4889 , piece_id=S2DNNAXK5YHKHBENEO2OD3JEYSILNFLEUCXH5O4ZBPZGWFCKRMMQ, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:50.456858006 (189h8m44.887938157s) [noise=true , address=45.91.116.14:4198 , piece_id=3QDHIGMC4BVOBCLR3NMSRGNXKB6T354BZBBXW7D6XGUIP5UYAEMA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:34:54.178960324 (189h8m48.610040455s) [noise=true , address=104.36.82.39:4734 , piece_id=NRFIPHK7CDGM33Y7XMO25TJY2Y3RY4V6L556HTKRTIMB2PI2FUJA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:35:03.464457833 (189h8m57.895537964s) [noise=true , address=77.90.31.15:4725 , piece_id=TD652IP6PITO3S7CGNEHIARRUQKV62CI4TCSVXVDBHZT33LMXUFA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: <nil> Aug 16 14:35:39.614950309 (189h9m34.04603044s) [noise=true , address=104.36.82.23:4474 , piece_id=Y3YM3UEB2DOAGZGX6IDEGWQP4PQPQWRFZOYQWHAIFNAAXKOMVSQQ, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: read tcp 188.166.225.170:52308->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615000916 (189h9m34.046081047s) [noise=true , address=45.91.116.13:4178 , piece_id=XLASEEFYZZCK2EHBDRISOXG6S5G6ITDXPDNEUNA5RMGPVFFW4ZEA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:125m3sbLAzXdKnjK9WZmasPccheZX63eDagUXqxVCPiakA9ApNX, address:45.91.116.13:4178): protocol: read tcp 188.166.225.170:39408->45.91.116.13:4178: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615034250 (189h9m34.046114381s) [noise=true , address=77.90.31.19:4857 , piece_id=UO3T4HPP7ELDIBODRDUVZABBFH4E2AQ4DMVTVNRP6QF2ZOTZJBRA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:12i57FF97oEddYLZpNxSq5or9WLEquL8ZW1cVUuW4AThBbDqcCL, address:77.90.31.19:4857): protocol: read tcp 188.166.225.170:37302->77.90.31.19:4857: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615089225 (189h9m34.046169356s) [noise=true , address=104.36.82.38:4704 , piece_id=UWTC6MN2PUIHJDO7LUXXU6VU5E6AWHTFR56J3GVUMKRV3X5WXZRA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1225CoSQvGuiWjJZWaXhvvq4gxzMymZqUb9fBrfLWw4RekwAEhp, address:104.36.82.38:4704): protocol: write tcp 188.166.225.170:60564->104.36.82.38:4704: use of closed network connection; write tcp 188.166.225.170:60564->104.36.82.38:4704: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60564->104.36.82.38:4704: use of closed network connection Aug 16 14:35:39.615113432 (189h9m34.046193553s) [noise=true , address=77.90.31.14:4690 , piece_id=UUAPEVRI2NDXSM3DIE2B4T2B3I5TRUKVFZS7S6KDOYEXGTUSIUYA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: read tcp 188.166.225.170:50794->77.90.31.14:4690: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615120976 (189h9m34.046201127s) [noise=true , address=77.90.31.13:4648 , piece_id=AAOHQWQC6BQX6SBBKVSC7VR3GA5UYIRBULI4YK7S5KPD7CQKPOWQ, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1niTsTaHjA6LoMMrkEsSnLcRDhZtVAjxsbXUXVeXzhUp3FPbWh, address:77.90.31.13:4648): protocol: read tcp 188.166.225.170:36442->77.90.31.13:4648: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615171973 (189h9m34.046252125s) [noise=true , address=77.90.31.12:4602 , piece_id=TX5WQHHAX5I6F6F7ZDEP7GNB4E5SGFMVICWXQGCXRVNICFT6D7AA, seg_pos={0 0}, upload=196789]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: read tcp 188.166.225.170:42998->77.90.31.12:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:39.615216529 (189h9m34.04629671s) [seg_pos={0 0}, upload=196789]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Aug 16 14:35:41.235994622 (189h9m35.667074743s) [upload=196789]: Done uploading segments... Aug 16 14:35:41.236021373 (189h9m35.667101504s) [upload=196789]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m1.623033095s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.234560785s FAIL (2025-08-16 14:35:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== QQ7WTQUAHTJ5TXV75E733HJFA633P5JRA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.951628041 (189h8m29.382708172s) [noise=true , address=66.23.193.175:4563 , piece_id=5GG45NDIMJ7IS6VN5ZXVDEK4AJHXWYM3NYECQ57UL53KI4KZ7DBA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.951836219 (189h8m29.38291635s) [noise=true , address=104.36.82.2:4753 , piece_id=375FY6WK5JXQ2COG6IGBKC7PSM2ANELMJM6ZTXUHSDBSXGRUKGQQ, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952052283 (189h8m29.383132414s) [noise=true , address=104.36.82.44:4882 , piece_id=PKPB7J6QD4W454FMSJGARPXEPVFDASVWMS5443OZ7KIJIIVEVYVA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952254570 (189h8m29.383334701s) [noise=true , address=104.36.82.3:4769 , piece_id=J3AJIDJZJLCBKG2GCTINZBJIN32Y6KOZWQTOTP6DQKOLX6ERLE6A, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952498887 (189h8m29.383579008s) [noise=true , address=77.90.31.13:4655 , piece_id=VRYJ2VRUAHTINO53SJI774HW47S3OEGUGSQIY57D34BESSUTKNBQ, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952698509 (189h8m29.38377864s) [noise=true , address=77.90.31.16:4764 , piece_id=DJPW6AZMK4P43AL4UZG7MP6UCXCK2K72TS33VD4JYNG4WH535HLA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.952902660 (189h8m29.383982781s) [noise=true , address=77.90.31.12:4619 , piece_id=L3RZOAULMD4AVOG2NIB3RU44IUPCXIURMPQJCRZBIVNJN2YVIDNA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:34.953096290 (189h8m29.384176422s) [noise=true , address=77.90.31.11:4585 , piece_id=MLXWH6KZS6ZDST4DZFI3QC652LWUNQNJ5SMS32UQS65TDWYILBHA, seg_pos={0 0}, upload=196785]: Uploading piece... Aug 16 14:34:36.088091971 (189h8m30.519172102s) [upload=196785]: Next returned nil segment Aug 16 14:34:36.088130115 (189h8m30.519210246s) [upload=196785]: Waiting for error group managing segments... Aug 16 14:34:36.405455542 (189h8m30.836535673s) [noise=true , address=66.23.193.175:4563 , piece_id=5GG45NDIMJ7IS6VN5ZXVDEK4AJHXWYM3NYECQ57UL53KI4KZ7DBA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:36.792259310 (189h8m31.223339431s) [noise=true , address=79.127.220.101:4073 , piece_id=H6IU6XQGKLLNEU7WSYKY2XA5G7VMOUH6KZ65BIQIQLUKBVJUPLHA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:37.146333052 (189h8m31.577413173s) [noise=true , address=79.127.201.215:4346 , piece_id=DA7IVKEOBVCOKUHBUMJY2M2IS2IUE7WK34MATIN6WTET5P5R5PVA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:38.678171978 (189h8m33.109252099s) [noise=true , address=77.90.31.13:4655 , piece_id=VRYJ2VRUAHTINO53SJI774HW47S3OEGUGSQIY57D34BESSUTKNBQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:55.651073853 (189h8m50.082153984s) [noise=true , address=104.36.82.38:4712 , piece_id=4JQB47DETHX7OY3W5HOKTWMAF6Q577ZFWOTHGPMLQOS5MTPFGO6A, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:34:57.241163206 (189h8m51.672243408s) [noise=true , address=104.36.82.59:4309 , piece_id=LPJ3RMEXVALSGSBWWGLSSGWY4YOAUNXPT2QGSHXIRBPG66BVUCXQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: <nil> Aug 16 14:35:33.848810590 (189h9m28.279890711s) [noise=true , address=77.90.31.18:4834 , piece_id=XQ3TGTB3Q2W6QRL7A5SWQ3CTYFB46CXARY6IOAWGSXB4QAB5N64Q, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:50154->77.90.31.18:4834: use of closed network connection; write tcp 188.166.225.170:50154->77.90.31.18:4834: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50154->77.90.31.18:4834: use of closed network connection Aug 16 14:35:33.848868651 (189h9m28.279948772s) [noise=true , address=104.36.82.44:4882 , piece_id=PKPB7J6QD4W454FMSJGARPXEPVFDASVWMS5443OZ7KIJIIVEVYVA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1TU7roqQsKNGvNeqmWdVxMregk9Dqn3YZpfdvK6xp7c3zYvccc, address:104.36.82.44:4882): protocol: write tcp 188.166.225.170:53334->104.36.82.44:4882: use of closed network connection; write tcp 188.166.225.170:53334->104.36.82.44:4882: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53334->104.36.82.44:4882: use of closed network connection Aug 16 14:35:33.848905823 (189h9m28.279985954s) [noise=true , address=104.36.82.3:4769 , piece_id=J3AJIDJZJLCBKG2GCTINZBJIN32Y6KOZWQTOTP6DQKOLX6ERLE6A, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:104.36.82.3:4769): protocol: read tcp 188.166.225.170:43434->104.36.82.3:4769: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.848933035 (189h9m28.280013166s) [noise=true , address=77.90.31.12:4619 , piece_id=L3RZOAULMD4AVOG2NIB3RU44IUPCXIURMPQJCRZBIVNJN2YVIDNA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:12Y3X8z9oBzPFDeM4psz1KBtFTcJQS4STc4xiL2N2V8sCEcDDKX, address:77.90.31.12:4619): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 14:35:33.848947472 (189h9m28.280027603s) [noise=true , address=104.36.82.60:4365 , piece_id=7XG5RETE5D4XCRNDTEZSDEGXJ4PKUEXRH65F4I2LUSLORWN4JOUQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: read tcp 188.166.225.170:33404->104.36.82.60:4365: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849021173 (189h9m28.280101304s) [noise=true , address=77.90.31.16:4764 , piece_id=DJPW6AZMK4P43AL4UZG7MP6UCXCK2K72TS33VD4JYNG4WH535HLA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT, address:77.90.31.16:4764): protocol: write tcp 188.166.225.170:49000->77.90.31.16:4764: use of closed network connection; write tcp 188.166.225.170:49000->77.90.31.16:4764: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49000->77.90.31.16:4764: use of closed network connection Aug 16 14:35:33.849117999 (189h9m28.28019813s) [noise=true , address=77.90.31.11:4585 , piece_id=MLXWH6KZS6ZDST4DZFI3QC652LWUNQNJ5SMS32UQS65TDWYILBHA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1Tve8WUWhJCK2cVWLbxxnz3WVzR2bi2wLx5XrF6QbhSN4HcsQG, address:77.90.31.11:4585): protocol: write tcp 188.166.225.170:56128->77.90.31.11:4585: use of closed network connection; write tcp 188.166.225.170:56128->77.90.31.11:4585: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56128->77.90.31.11:4585: use of closed network connection Aug 16 14:35:33.849161221 (189h9m28.280241342s) [noise=true , address=77.90.31.19:4875 , piece_id=7LF3JKR5YQO2IAKOCC6QQ7WTQUAHTJ5TXV75E733HJFA633P5JRA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1xSFQaW3aVnAVDYE5r5ADmCGU95vWnshuZyte32W2YAj94avNn, address:77.90.31.19:4875): protocol: write tcp 188.166.225.170:38064->77.90.31.19:4875: use of closed network connection; write tcp 188.166.225.170:38064->77.90.31.19:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38064->77.90.31.19:4875: use of closed network connection Aug 16 14:35:33.849200948 (189h9m28.280281079s) [noise=true , address=104.36.82.5:4843 , piece_id=JZOHQO4YX7227CIM6TGAQ7QJ4JJIUVE2YC5RZ6UIDTYMOESTWMAQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS, address:104.36.82.5:4843): protocol: read tcp 188.166.225.170:35936->104.36.82.5:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849300387 (189h9m28.280380518s) [noise=true , address=104.36.82.2:4753 , piece_id=375FY6WK5JXQ2COG6IGBKC7PSM2ANELMJM6ZTXUHSDBSXGRUKGQQ, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:104.36.82.2:4753): protocol: read tcp 188.166.225.170:45972->104.36.82.2:4753: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849333711 (189h9m28.280413842s) [noise=true , address=104.36.82.39:4760 , piece_id=IQ3MTMPLF4TUASFFY3U2O325XK5VZG7MK3P2HLEKOSBKFVV64S5Q, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1jJcppBK9zJzGedgY9KEjEA8twqtkdZcjhcnXJoN5AWi5QzR4E, address:104.36.82.39:4760): protocol: read tcp 188.166.225.170:41552->104.36.82.39:4760: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849343871 (189h9m28.280424002s) [noise=true , address=104.36.82.18:4389 , piece_id=YMIA2XHWMZZRHULDAP7VLTMU5COOQOMAYLSCAWZ2RVH6FAHGZNVA, seg_pos={0 0}, upload=196785]: Done uploading piece. err: ecclient: upload failed (node:1mMqCXPNzFx4cnu8Nb3gNKcJFJ4tGJjnkwY2EVPjUBnwNVcM9R, address:104.36.82.18:4389): protocol: read tcp 188.166.225.170:60738->104.36.82.18:4389: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:33.849392264 (189h9m28.280472395s) [seg_pos={0 0}, upload=196785]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Aug 16 14:35:34.082190547 (189h9m28.513270678s) [upload=196785]: Done uploading segments... Aug 16 14:35:34.082221266 (189h9m28.513301397s) [upload=196785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.234581837s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 11.47648151s FAIL (2025-08-16 14:35:11 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 11.47650681s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.231753857s FAIL (2025-08-16 14:35:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== IWLI2DC77MJ23LBPVVMRFJ6S3YUIVX723OTSWE5N4Q, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.444967901 (189h8m35.876048033s) [noise=true , address=104.36.82.10:4078 , piece_id=FP4RP4CK2X754SJB5ENID6JWX7IJGYRNQSVMIUDQKAIG3M4E6TXA, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.445254930 (189h8m35.876335051s) [noise=true , address=77.90.31.13:4628 , piece_id=BQUXRM6AO4DFV3CNZA5TKEZRRJEN2KPOBSUJ6Y3LJSJGUAOOVQNA, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.445275830 (189h8m35.876355962s) [noise=true , address=104.36.82.2:4732 , piece_id=VIPZJYLPDTGCHMNHHHY5LGDSFA5DZXG5Q2CFCUGVOGTVHO7ILQ2Q, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.445586715 (189h8m35.876666836s) [noise=true , address=77.90.31.21:4879 , piece_id=H7HGHNEVGAZ6KJGXIE2JC4H7XEY4USOLQJWIOWAUQYAXMCTSLZ7Q, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.445633465 (189h8m35.876713596s) [noise=true , address=104.36.82.6:4873 , piece_id=TQN3QFX2DBS5DZICBRYQ3VHPMBXTE54IASMGPABQP6D6PH4ROVLQ, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.445889043 (189h8m35.876969174s) [noise=true , address=104.36.82.9:4007 , piece_id=MDLI7SRMJ3AMON7BF6Z7WXAOG3KDEDNBL3WW2OOFDVC6Y4G77ZCA, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.445907459 (189h8m35.87698759s) [noise=true , address=104.36.82.12:4150 , piece_id=UXUR2NKX5R7POBV4TV7UFWP5C6B7HZPI3M6KPBHHQA64O7S2NA6Q, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.445941995 (189h8m35.877022126s) [noise=true , address=45.91.116.12:4227 , piece_id=U3XLTIMFON4KRNOBU3QTZHWOBCMCMVTUM6VOSZWRV76BBUJE3X5Q, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.446181493 (189h8m35.877261624s) [noise=true , address=104.36.82.15:4259 , piece_id=5FAMYGZKN56IWW27VD3TNNOHFJJIETHOHMMR4P6NK3NCTFVAHDGQ, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.446480926 (189h8m35.877561057s) [noise=true , address=104.36.82.6:4874 , piece_id=KA6GCQKUA24AKJX72JUAWTVJB7DIKDHQIIGIH6RM6DYAGCRJZAVQ, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.446683003 (189h8m35.877763134s) [noise=true , address=77.90.31.14:4663 , piece_id=7QIGKD4X753T3WDVWMJTYVXXJ2Y7S6GOLTKUOJNWDWMXICDLRNJA, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.446896852 (189h8m35.877976983s) [noise=true , address=45.91.116.5:4299 , piece_id=5NCTI3LQHG3WXKNEHWOCJ2Y4FYFTULNOUGQ33QHI4VFKYHEGZ4XA, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:41.447088889 (189h8m35.87816901s) [noise=true , address=104.36.82.16:4294 , piece_id=YTDTXNNZAJSNE4Z4DPPG3F6R6ZDEWTT7PQWX33JN5ZNCFKOE72LQ, seg_pos={0 0}, upload=196790]: Uploading piece... Aug 16 14:34:43.302465390 (189h8m37.733545521s) [upload=196790]: Next returned nil segment Aug 16 14:34:43.302528351 (189h8m37.733608502s) [upload=196790]: Waiting for error group managing segments... Aug 16 14:34:43.613294488 (189h8m38.044374609s) [noise=true , address=104.36.82.12:4150 , piece_id=UXUR2NKX5R7POBV4TV7UFWP5C6B7HZPI3M6KPBHHQA64O7S2NA6Q, seg_pos={0 0}, upload=196790]: Done uploading piece. err: <nil> Aug 16 14:34:46.392336053 (189h8m40.823416184s) [noise=true , address=104.36.82.4:4804 , piece_id=T2B4UAZNM7IWLI2DC77MJ23LBPVVMRFJ6S3YUIVX723OTSWE5N4Q, seg_pos={0 0}, upload=196790]: Done uploading piece. err: <nil> Aug 16 14:34:46.891161411 (189h8m41.322241532s) [noise=true , address=104.36.82.9:4007 , piece_id=MDLI7SRMJ3AMON7BF6Z7WXAOG3KDEDNBL3WW2OOFDVC6Y4G77ZCA, seg_pos={0 0}, upload=196790]: Done uploading piece. err: <nil> Aug 16 14:34:48.283390080 (189h8m42.714470211s) [noise=true , address=104.36.82.15:4259 , piece_id=5FAMYGZKN56IWW27VD3TNNOHFJJIETHOHMMR4P6NK3NCTFVAHDGQ, seg_pos={0 0}, upload=196790]: Done uploading piece. err: <nil> Aug 16 14:35:10.499093155 (189h9m4.930173286s) [noise=true , address=104.36.82.2:4732 , piece_id=VIPZJYLPDTGCHMNHHHY5LGDSFA5DZXG5Q2CFCUGVOGTVHO7ILQ2Q, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: read tcp 188.166.225.170:43178->104.36.82.2:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499139373 (189h9m4.930219504s) [noise=true , address=104.36.82.10:4078 , piece_id=FP4RP4CK2X754SJB5ENID6JWX7IJGYRNQSVMIUDQKAIG3M4E6TXA, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: read tcp 188.166.225.170:43970->104.36.82.10:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499175362 (189h9m4.930255494s) [noise=true , address=104.36.82.6:4874 , piece_id=KA6GCQKUA24AKJX72JUAWTVJB7DIKDHQIIGIH6RM6DYAGCRJZAVQ, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: read tcp 188.166.225.170:48246->104.36.82.6:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499208846 (189h9m4.930288977s) [noise=true , address=45.91.116.5:4299 , piece_id=5NCTI3LQHG3WXKNEHWOCJ2Y4FYFTULNOUGQ33QHI4VFKYHEGZ4XA, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 188.166.225.170:45292->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499241979 (189h9m4.930322111s) [noise=true , address=104.36.82.15:4257 , piece_id=HYMMQP6JEIMUHHSTY3NVSTKZ7NB64QL7SKSKUDAEVXLMJKWKLWAQ, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: read tcp 188.166.225.170:54188->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499265695 (189h9m4.930345826s) [noise=true , address=45.91.116.12:4227 , piece_id=U3XLTIMFON4KRNOBU3QTZHWOBCMCMVTUM6VOSZWRV76BBUJE3X5Q, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 188.166.225.170:42562->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499294620 (189h9m4.930374751s) [noise=true , address=77.90.31.14:4663 , piece_id=7QIGKD4X753T3WDVWMJTYVXXJ2Y7S6GOLTKUOJNWDWMXICDLRNJA, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 188.166.225.170:50772->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499339106 (189h9m4.930419237s) [noise=true , address=77.90.31.21:4879 , piece_id=H7HGHNEVGAZ6KJGXIE2JC4H7XEY4USOLQJWIOWAUQYAXMCTSLZ7Q, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 188.166.225.170:37454->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499364985 (189h9m4.930445116s) [noise=true , address=104.36.82.16:4294 , piece_id=YTDTXNNZAJSNE4Z4DPPG3F6R6ZDEWTT7PQWX33JN5ZNCFKOE72LQ, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: read tcp 188.166.225.170:42050->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499391586 (189h9m4.930471717s) [noise=true , address=104.36.82.6:4873 , piece_id=TQN3QFX2DBS5DZICBRYQ3VHPMBXTE54IASMGPABQP6D6PH4ROVLQ, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: read tcp 188.166.225.170:34024->104.36.82.6:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499409740 (189h9m4.930489871s) [noise=true , address=104.36.82.11:4115 , piece_id=7L52YTTLTLNBL34YYIURBYSJKYTZI6JHC2NGR7KTDLLUE6DCWCVQ, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: read tcp 188.166.225.170:45442->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499444177 (189h9m4.930524308s) [noise=true , address=77.90.31.13:4628 , piece_id=BQUXRM6AO4DFV3CNZA5TKEZRRJEN2KPOBSUJ6Y3LJSJGUAOOVQNA, seg_pos={0 0}, upload=196790]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 188.166.225.170:58128->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 14:35:10.499459366 (189h9m4.930539497s) [seg_pos={0 0}, upload=196790]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Aug 16 14:35:10.729290032 (189h9m5.160370213s) [upload=196790]: Done uploading segments... Aug 16 14:35:10.729321522 (189h9m5.160401693s) [upload=196790]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.23178078s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................X........XX] duration: 17.578780443s FAIL (2025-08-16 14:35:04 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 17.578807035s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................X...............X..........................] duration: 2.45067621s FAIL (2025-08-16 14:35:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 16 14:15:15.081221025 (188h49m9.512301156s) [upload=196448]: Starting upload Aug 16 14:15:15.081268716 (188h49m9.512348847s) [upload=196448]: Done starting upload Aug 16 14:15:15.081327208 (188h49m9.512407339s) [upload=196448]: Uploading segments... Aug 16 14:15:15.081365671 (188h49m9.512445802s) [upload=196448]: Got next segment. Inline: true Aug 16 14:15:15.081369339 (188h49m9.51244946s) [upload=196448]: Waiting for error group managing segments... Aug 16 14:15:15.081400809 (188h49m9.51248094s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:15:15.081404686 (188h49m9.512484817s) [upload=196448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:15:15.081407071 (188h49m9.512487202s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:15:16.556926721 (188h49m10.988006852s) [upload=196448]: Done uploading segments... Aug 16 14:15:16.556955035 (188h49m10.988035166s) [upload=196448]: Upload finished. err: <nil> Duration:* 2.450702009s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X........X] duration: 10.243087845s FAIL (2025-08-16 14:34:38 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 10.243114815s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.......X...X...........X................................] duration: 9.690760162s FAIL (2025-08-16 14:34:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 16 14:06:47.349724332 (188h40m41.780804454s) [upload=196301]: Starting upload Aug 16 14:06:47.349762074 (188h40m41.780842205s) [upload=196301]: Done starting upload Aug 16 14:06:47.349918454 (188h40m41.780998575s) [upload=196301]: Uploading segments... Aug 16 14:06:47.349959432 (188h40m41.781039563s) [upload=196301]: Got next segment. Inline: false Aug 16 14:06:47.349980362 (188h40m41.781060493s) [upload=196301]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:06:47.349984650 (188h40m41.781064771s) [upload=196301]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 14:06:48.041961874 (188h40m42.473041995s) [seg_pos={0 0}, upload=196301]: Begin upload segment... Aug 16 14:06:48.042016458 (188h40m42.473096589s) [seg_pos={0 0}, upload=196301]: Done begin upload segment. Aug 16 14:06:48.042084658 (188h40m42.473164789s) [noise=true , address=104.36.82.39:4747 , piece_id=5ZF7PJCPLF274WV6SENDN5YQCALUUEXLH4DAL4VFDRHYP4Y35DLA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.042108744 (188h40m42.473189016s) [noise=true , address=45.91.116.12:4239 , piece_id=YKKGHYYLFH6D7K5SGRK7VXQOH37SWR4WJVYAY457JYCWBWFM6GIQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.042556851 (188h40m42.473636982s) [noise=true , address=104.36.82.44:4887 , piece_id=7I34ZNQAFY3GDVRMDC3T5MDHHZ3HHLIOF2554JU3S4KRPE6TVVMA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.042743739 (188h40m42.47382387s) [noise=true , address=66.23.193.175:4557 , piece_id=EMJ4TIJPUPKAYDOW64SGYSYVSJOD2XWKC33PGTFQMKCY5KJ3N6UA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.042878577 (188h40m42.473958708s) [noise=true , address=104.36.82.23:4453 , piece_id=MTWTEPYP2HT4MEK2JSHTOU6LRY5QT4CCYUWFL6TI4IQVW3QWRVMA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.043057941 (188h40m42.474138062s) [noise=true , address=104.36.82.60:4365 , piece_id=Q26BVA4HZUQD2LN2B63JC3SI5SWNSJ5632XYRVDWXT3TXETF3H6A, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.043178601 (188h40m42.474258732s) [noise=true , address=77.90.31.19:4849 , piece_id=M4SHIBE3WX47WNDT2QAPPWJ4TVBXD3WGP33IWKQCABKFH7MFFCPA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.043469949 (188h40m42.47455008s) [noise=true , address=104.36.82.5:4853 , piece_id=BHTB5IEENVV5ASX47CAARY575RSNA3S2AGM6UPLK2SAROM4WVWXA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.044068574 (188h40m42.475148705s) [noise=true , address=104.36.82.10:4107 , piece_id=M3DYFJTZDFJIZXXNQSOMFGM3H6AEC6YM276NOOFY6VBBA6S7ADLQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.044318653 (188h40m42.475398774s) [noise=true , address=45.91.116.4:4958 , piece_id=PUSXFXZXNT3KMN3AVNGF75WAB4W65WQEQWMZTSLD4ZKMSONV6Q4Q, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.044813640 (188h40m42.475893771s) [noise=true , address=77.90.31.16:4759 , piece_id=PRPI3NCJ4SEII523WSJLDMZSB7L5NZYPAPBMFY35JPOEZG5HTPZQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.045182145 (188h40m42.476262276s) [noise=true , address=104.36.82.59:4312 , piece_id=SSDD5GOKZ57C26ROC43YETFUNLFD57ZV2OYPG3EORBQIHWJL365Q, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.045549167 (188h40m42.476629298s) [noise=true , address=77.90.31.13:4652 , piece_id=PVMSTPXQ3RF6BTR4N4YWQM23R3H4HGCW5ES6JHLDTYDM5M74VF6Q, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046068210 (188h40m42.477148341s) [noise=true , address=77.90.31.11:4557 , piece_id=Y6YQNWEDNIPIQHGCWJPS54JK3D2NJAZPMRXLNDGGH4JBCMBEPLNA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046291598 (188h40m42.477371729s) [noise=true , address=79.127.220.101:4071 , piece_id=ALQRRGXITKOFKWAOLR2MOAZD7HDAKPDLXDJRSUOADOL3MXGTCTGQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046502321 (188h40m42.477582452s) [noise=true , address=79.127.201.215:4344 , piece_id=ZX7JBR5CWDMVVHRBTBT5HDVXTAPS2W76H5GIHXQRBG2SG6HXILWQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046700249 (188h40m42.47778037s) [noise=true , address=79.127.205.234:4617 , piece_id=GM62VNXOHMZAJFGAE4VUL5T3OCMFWBNUW274YOVGOLUKXCLIOE7A, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046891105 (188h40m42.477971236s) [noise=true , address=104.36.82.2:4760 , piece_id=5UGVFTM7JZQ4HRHJB6JFUTI7VX6TKHX2ZWBSKE2L7OEIH2IXEHUQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:49.189586568 (188h40m43.620666689s) [upload=196301]: Next returned nil segment Aug 16 14:06:49.189605764 (188h40m43.620685895s) [upload=196301]: Waiting for error group managing segments... Aug 16 14:06:49.481177906 (188h40m43.912258027s) [noise=true , address=79.127.205.234:4617 , piece_id=GM62VNXOHMZAJFGAE4VUL5T3OCMFWBNUW274YOVGOLUKXCLIOE7A, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:49.523286867 (188h40m43.954366998s) [noise=true , address=66.23.193.175:4557 , piece_id=EMJ4TIJPUPKAYDOW64SGYSYVSJOD2XWKC33PGTFQMKCY5KJ3N6UA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:49.973285570 (188h40m44.404365691s) [noise=true , address=104.36.82.2:4760 , piece_id=5UGVFTM7JZQ4HRHJB6JFUTI7VX6TKHX2ZWBSKE2L7OEIH2IXEHUQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:49.985931557 (188h40m44.417011688s) [noise=true , address=104.36.82.59:4312 , piece_id=SSDD5GOKZ57C26ROC43YETFUNLFD57ZV2OYPG3EORBQIHWJL365Q, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:49.997459735 (188h40m44.428539866s) [noise=true , address=79.127.220.101:4071 , piece_id=ALQRRGXITKOFKWAOLR2MOAZD7HDAKPDLXDJRSUOADOL3MXGTCTGQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.021024593 (188h40m44.452104714s) [noise=true , address=104.36.82.10:4107 , piece_id=M3DYFJTZDFJIZXXNQSOMFGM3H6AEC6YM276NOOFY6VBBA6S7ADLQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.027936639 (188h40m44.45901676s) [noise=true , address=104.36.82.5:4853 , piece_id=BHTB5IEENVV5ASX47CAARY575RSNA3S2AGM6UPLK2SAROM4WVWXA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.052542770 (188h40m44.483622901s) [noise=true , address=79.127.201.215:4344 , piece_id=ZX7JBR5CWDMVVHRBTBT5HDVXTAPS2W76H5GIHXQRBG2SG6HXILWQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.065183026 (188h40m44.496263158s) [noise=true , address=104.36.82.39:4747 , piece_id=5ZF7PJCPLF274WV6SENDN5YQCALUUEXLH4DAL4VFDRHYP4Y35DLA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.080131979 (188h40m44.51121211s) [noise=true , address=104.36.82.44:4887 , piece_id=7I34ZNQAFY3GDVRMDC3T5MDHHZ3HHLIOF2554JU3S4KRPE6TVVMA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.138131625 (188h40m44.569211756s) [noise=true , address=104.36.82.23:4453 , piece_id=MTWTEPYP2HT4MEK2JSHTOU6LRY5QT4CCYUWFL6TI4IQVW3QWRVMA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.191502545 (188h40m44.622582666s) [noise=true , address=77.90.31.19:4849 , piece_id=M4SHIBE3WX47WNDT2QAPPWJ4TVBXD3WGP33IWKQCABKFH7MFFCPA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.347285508 (188h40m44.778365629s) [noise=true , address=77.90.31.11:4557 , piece_id=Y6YQNWEDNIPIQHGCWJPS54JK3D2NJAZPMRXLNDGGH4JBCMBEPLNA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.521348719 (188h40m44.95242885s) [noise=true , address=45.91.116.12:4239 , piece_id=YKKGHYYLFH6D7K5SGRK7VXQOH37SWR4WJVYAY457JYCWBWFM6GIQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.521377143 (188h40m44.952457264s) [seg_pos={0 0}, upload=196301]: Segment reached optimal threshold of 14 pieces. Aug 16 14:06:50.521490380 (188h40m44.952570501s) [noise=true , address=77.90.31.13:4652 , piece_id=PVMSTPXQ3RF6BTR4N4YWQM23R3H4HGCW5ES6JHLDTYDM5M74VF6Q, seg_pos={0 0}, upload=196301]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 16 14:06:50.521596403 (188h40m44.952676534s) [noise=true , address=45.91.116.4:4958 , piece_id=PUSXFXZXNT3KMN3AVNGF75WAB4W65WQEQWMZTSLD4ZKMSONV6Q4Q, seg_pos={0 0}, upload=196301]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1PRN34y7cCWokJkxgu5Zk4eBjDPW5iddNefMKXvUkJUTAqkKve): protocol: expected piece hash; write tcp 188.166.225.170:36678->45.91.116.4:4958: use of closed network connection; write tcp 188.166.225.170:36678->45.91.116.4:4958: use of closed network connection; EOF Aug 16 14:06:50.521656798 (188h40m44.952736929s) [noise=true , address=77.90.31.16:4759 , piece_id=PRPI3NCJ4SEII523WSJLDMZSB7L5NZYPAPBMFY35JPOEZG5HTPZQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Aug 16 14:06:50.521674041 (188h40m44.952754162s) [noise=true , address=104.36.82.60:4365 , piece_id=Q26BVA4HZUQD2LN2B63JC3SI5SWNSJ5632XYRVDWXT3TXETF3H6A, seg_pos={0 0}, upload=196301]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a): protocol: expected piece hash; context canceled; EOF Aug 16 14:06:50.521688028 (188h40m44.952768159s) [seg_pos={0 0}, upload=196301]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 14:06:50.521747772 (188h40m44.952827903s) [upload=196301]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 14:06:50.521751780 (188h40m44.952831911s) [upload=196301]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:06:54.003684026 (188h40m48.434764167s) [upload=196301]: Done uploading segments... Aug 16 14:06:54.003703113 (188h40m48.434783244s) [upload=196301]: Upload finished. err: <nil> Duration: 9.690790662s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.290533359s FAIL (2025-08-16 14:34:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.290558137s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................X...............X.........................] duration: 2.130147166s FAIL (2025-08-16 14:33:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 16 14:15:15.081221025 (188h49m9.512301156s) [upload=196448]: Starting upload Aug 16 14:15:15.081268716 (188h49m9.512348847s) [upload=196448]: Done starting upload Aug 16 14:15:15.081327208 (188h49m9.512407339s) [upload=196448]: Uploading segments... Aug 16 14:15:15.081365671 (188h49m9.512445802s) [upload=196448]: Got next segment. Inline: true Aug 16 14:15:15.081369339 (188h49m9.51244946s) [upload=196448]: Waiting for error group managing segments... Aug 16 14:15:15.081400809 (188h49m9.51248094s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:15:15.081404686 (188h49m9.512484817s) [upload=196448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:15:15.081407071 (188h49m9.512487202s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:15:16.556926721 (188h49m10.988006852s) [upload=196448]: Done uploading segments... Aug 16 14:15:16.556955035 (188h49m10.988035166s) [upload=196448]: Upload finished. err: <nil> Duration:* 2.130173366s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.949218451s FAIL (2025-08-16 14:33:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.949238049s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X.......X...X...........X...............................] duration: 12.037003948s FAIL (2025-08-16 14:33:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 16 14:06:47.349724332 (188h40m41.780804454s) [upload=196301]: Starting upload Aug 16 14:06:47.349762074 (188h40m41.780842205s) [upload=196301]: Done starting upload Aug 16 14:06:47.349918454 (188h40m41.780998575s) [upload=196301]: Uploading segments... Aug 16 14:06:47.349959432 (188h40m41.781039563s) [upload=196301]: Got next segment. Inline: false Aug 16 14:06:47.349980362 (188h40m41.781060493s) [upload=196301]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:06:47.349984650 (188h40m41.781064771s) [upload=196301]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 14:06:48.041961874 (188h40m42.473041995s) [seg_pos={0 0}, upload=196301]: Begin upload segment... Aug 16 14:06:48.042016458 (188h40m42.473096589s) [seg_pos={0 0}, upload=196301]: Done begin upload segment. Aug 16 14:06:48.042084658 (188h40m42.473164789s) [noise=true , address=104.36.82.39:4747 , piece_id=5ZF7PJCPLF274WV6SENDN5YQCALUUEXLH4DAL4VFDRHYP4Y35DLA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.042108744 (188h40m42.473189016s) [noise=true , address=45.91.116.12:4239 , piece_id=YKKGHYYLFH6D7K5SGRK7VXQOH37SWR4WJVYAY457JYCWBWFM6GIQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.042556851 (188h40m42.473636982s) [noise=true , address=104.36.82.44:4887 , piece_id=7I34ZNQAFY3GDVRMDC3T5MDHHZ3HHLIOF2554JU3S4KRPE6TVVMA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.042743739 (188h40m42.47382387s) [noise=true , address=66.23.193.175:4557 , piece_id=EMJ4TIJPUPKAYDOW64SGYSYVSJOD2XWKC33PGTFQMKCY5KJ3N6UA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.042878577 (188h40m42.473958708s) [noise=true , address=104.36.82.23:4453 , piece_id=MTWTEPYP2HT4MEK2JSHTOU6LRY5QT4CCYUWFL6TI4IQVW3QWRVMA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.043057941 (188h40m42.474138062s) [noise=true , address=104.36.82.60:4365 , piece_id=Q26BVA4HZUQD2LN2B63JC3SI5SWNSJ5632XYRVDWXT3TXETF3H6A, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.043178601 (188h40m42.474258732s) [noise=true , address=77.90.31.19:4849 , piece_id=M4SHIBE3WX47WNDT2QAPPWJ4TVBXD3WGP33IWKQCABKFH7MFFCPA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.043469949 (188h40m42.47455008s) [noise=true , address=104.36.82.5:4853 , piece_id=BHTB5IEENVV5ASX47CAARY575RSNA3S2AGM6UPLK2SAROM4WVWXA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.044068574 (188h40m42.475148705s) [noise=true , address=104.36.82.10:4107 , piece_id=M3DYFJTZDFJIZXXNQSOMFGM3H6AEC6YM276NOOFY6VBBA6S7ADLQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.044318653 (188h40m42.475398774s) [noise=true , address=45.91.116.4:4958 , piece_id=PUSXFXZXNT3KMN3AVNGF75WAB4W65WQEQWMZTSLD4ZKMSONV6Q4Q, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.044813640 (188h40m42.475893771s) [noise=true , address=77.90.31.16:4759 , piece_id=PRPI3NCJ4SEII523WSJLDMZSB7L5NZYPAPBMFY35JPOEZG5HTPZQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.045182145 (188h40m42.476262276s) [noise=true , address=104.36.82.59:4312 , piece_id=SSDD5GOKZ57C26ROC43YETFUNLFD57ZV2OYPG3EORBQIHWJL365Q, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.045549167 (188h40m42.476629298s) [noise=true , address=77.90.31.13:4652 , piece_id=PVMSTPXQ3RF6BTR4N4YWQM23R3H4HGCW5ES6JHLDTYDM5M74VF6Q, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046068210 (188h40m42.477148341s) [noise=true , address=77.90.31.11:4557 , piece_id=Y6YQNWEDNIPIQHGCWJPS54JK3D2NJAZPMRXLNDGGH4JBCMBEPLNA, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046291598 (188h40m42.477371729s) [noise=true , address=79.127.220.101:4071 , piece_id=ALQRRGXITKOFKWAOLR2MOAZD7HDAKPDLXDJRSUOADOL3MXGTCTGQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046502321 (188h40m42.477582452s) [noise=true , address=79.127.201.215:4344 , piece_id=ZX7JBR5CWDMVVHRBTBT5HDVXTAPS2W76H5GIHXQRBG2SG6HXILWQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046700249 (188h40m42.47778037s) [noise=true , address=79.127.205.234:4617 , piece_id=GM62VNXOHMZAJFGAE4VUL5T3OCMFWBNUW274YOVGOLUKXCLIOE7A, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:48.046891105 (188h40m42.477971236s) [noise=true , address=104.36.82.2:4760 , piece_id=5UGVFTM7JZQ4HRHJB6JFUTI7VX6TKHX2ZWBSKE2L7OEIH2IXEHUQ, seg_pos={0 0}, upload=196301]: Uploading piece... Aug 16 14:06:49.189586568 (188h40m43.620666689s) [upload=196301]: Next returned nil segment Aug 16 14:06:49.189605764 (188h40m43.620685895s) [upload=196301]: Waiting for error group managing segments... Aug 16 14:06:49.481177906 (188h40m43.912258027s) [noise=true , address=79.127.205.234:4617 , piece_id=GM62VNXOHMZAJFGAE4VUL5T3OCMFWBNUW274YOVGOLUKXCLIOE7A, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:49.523286867 (188h40m43.954366998s) [noise=true , address=66.23.193.175:4557 , piece_id=EMJ4TIJPUPKAYDOW64SGYSYVSJOD2XWKC33PGTFQMKCY5KJ3N6UA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:49.973285570 (188h40m44.404365691s) [noise=true , address=104.36.82.2:4760 , piece_id=5UGVFTM7JZQ4HRHJB6JFUTI7VX6TKHX2ZWBSKE2L7OEIH2IXEHUQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:49.985931557 (188h40m44.417011688s) [noise=true , address=104.36.82.59:4312 , piece_id=SSDD5GOKZ57C26ROC43YETFUNLFD57ZV2OYPG3EORBQIHWJL365Q, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:49.997459735 (188h40m44.428539866s) [noise=true , address=79.127.220.101:4071 , piece_id=ALQRRGXITKOFKWAOLR2MOAZD7HDAKPDLXDJRSUOADOL3MXGTCTGQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.021024593 (188h40m44.452104714s) [noise=true , address=104.36.82.10:4107 , piece_id=M3DYFJTZDFJIZXXNQSOMFGM3H6AEC6YM276NOOFY6VBBA6S7ADLQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.027936639 (188h40m44.45901676s) [noise=true , address=104.36.82.5:4853 , piece_id=BHTB5IEENVV5ASX47CAARY575RSNA3S2AGM6UPLK2SAROM4WVWXA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.052542770 (188h40m44.483622901s) [noise=true , address=79.127.201.215:4344 , piece_id=ZX7JBR5CWDMVVHRBTBT5HDVXTAPS2W76H5GIHXQRBG2SG6HXILWQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.065183026 (188h40m44.496263158s) [noise=true , address=104.36.82.39:4747 , piece_id=5ZF7PJCPLF274WV6SENDN5YQCALUUEXLH4DAL4VFDRHYP4Y35DLA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.080131979 (188h40m44.51121211s) [noise=true , address=104.36.82.44:4887 , piece_id=7I34ZNQAFY3GDVRMDC3T5MDHHZ3HHLIOF2554JU3S4KRPE6TVVMA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.138131625 (188h40m44.569211756s) [noise=true , address=104.36.82.23:4453 , piece_id=MTWTEPYP2HT4MEK2JSHTOU6LRY5QT4CCYUWFL6TI4IQVW3QWRVMA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.191502545 (188h40m44.622582666s) [noise=true , address=77.90.31.19:4849 , piece_id=M4SHIBE3WX47WNDT2QAPPWJ4TVBXD3WGP33IWKQCABKFH7MFFCPA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.347285508 (188h40m44.778365629s) [noise=true , address=77.90.31.11:4557 , piece_id=Y6YQNWEDNIPIQHGCWJPS54JK3D2NJAZPMRXLNDGGH4JBCMBEPLNA, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.521348719 (188h40m44.95242885s) [noise=true , address=45.91.116.12:4239 , piece_id=YKKGHYYLFH6D7K5SGRK7VXQOH37SWR4WJVYAY457JYCWBWFM6GIQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: <nil> Aug 16 14:06:50.521377143 (188h40m44.952457264s) [seg_pos={0 0}, upload=196301]: Segment reached optimal threshold of 14 pieces. Aug 16 14:06:50.521490380 (188h40m44.952570501s) [noise=true , address=77.90.31.13:4652 , piece_id=PVMSTPXQ3RF6BTR4N4YWQM23R3H4HGCW5ES6JHLDTYDM5M74VF6Q, seg_pos={0 0}, upload=196301]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KrMw5DoiMSiBHMZJ8Zf8ir7UnTzBwUt9XMhw3Ria6ASGkm3FA): protocol: expected piece hash; context canceled; EOF Aug 16 14:06:50.521596403 (188h40m44.952676534s) [noise=true , address=45.91.116.4:4958 , piece_id=PUSXFXZXNT3KMN3AVNGF75WAB4W65WQEQWMZTSLD4ZKMSONV6Q4Q, seg_pos={0 0}, upload=196301]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1PRN34y7cCWokJkxgu5Zk4eBjDPW5iddNefMKXvUkJUTAqkKve): protocol: expected piece hash; write tcp 188.166.225.170:36678->45.91.116.4:4958: use of closed network connection; write tcp 188.166.225.170:36678->45.91.116.4:4958: use of closed network connection; EOF Aug 16 14:06:50.521656798 (188h40m44.952736929s) [noise=true , address=77.90.31.16:4759 , piece_id=PRPI3NCJ4SEII523WSJLDMZSB7L5NZYPAPBMFY35JPOEZG5HTPZQ, seg_pos={0 0}, upload=196301]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13KqQ6GUrwQMP2ov1YRC1RJ6YqJhxB2T8MwxFEW1dFJADhBfYE): protocol: expected piece hash; context canceled; EOF Aug 16 14:06:50.521674041 (188h40m44.952754162s) [noise=true , address=104.36.82.60:4365 , piece_id=Q26BVA4HZUQD2LN2B63JC3SI5SWNSJ5632XYRVDWXT3TXETF3H6A, seg_pos={0 0}, upload=196301]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a): protocol: expected piece hash; context canceled; EOF Aug 16 14:06:50.521688028 (188h40m44.952768159s) [seg_pos={0 0}, upload=196301]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 16 14:06:50.521747772 (188h40m44.952827903s) [upload=196301]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 14:06:50.521751780 (188h40m44.952831911s) [upload=196301]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:06:54.003684026 (188h40m48.434764167s) [upload=196301]: Done uploading segments... Aug 16 14:06:54.003703113 (188h40m48.434783244s) [upload=196301]: Upload finished. err: <nil> Duration: 12.037039024s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................X...............X........................] duration: 2.737733716s FAIL (2025-08-16 14:33:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 16 14:15:15.081221025 (188h49m9.512301156s) [upload=196448]: Starting upload Aug 16 14:15:15.081268716 (188h49m9.512348847s) [upload=196448]: Done starting upload Aug 16 14:15:15.081327208 (188h49m9.512407339s) [upload=196448]: Uploading segments... Aug 16 14:15:15.081365671 (188h49m9.512445802s) [upload=196448]: Got next segment. Inline: true Aug 16 14:15:15.081369339 (188h49m9.51244946s) [upload=196448]: Waiting for error group managing segments... Aug 16 14:15:15.081400809 (188h49m9.51248094s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 14:15:15.081404686 (188h49m9.512484817s) [upload=196448]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 14:15:15.081407071 (188h49m9.512487202s) [upload=196448]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 14:15:16.556926721 (188h49m10.988006852s) [upload=196448]: Done uploading segments... Aug 16 14:15:16.556955035 (188h49m10.988035166s) [upload=196448]: Upload finished. err: <nil> Duration:* 2.737758422s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)