Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 58.330775122s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:32:08 UTC) orbiter warning: context deadline exceeded Duration: 58.33080134s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X..........XX....] duration: 21.614069746s trace: https://jaeger.dev.storj.io/trace/a0eab84190ca8f8 FAIL (2026-01-09 10:32:02 UTC) orbiter warning: context deadline exceeded Duration: 21.614099443s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 50 out of 60 is failed (threshold=6) [XX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX.XXXXXXX] duration: 5.000963213s trace: https://jaeger.dev.storj.io/trace/76c768fcd1a2633e FAIL (2026-01-09 10:31:42 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000984182s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X..................X..................................] duration: 5.755665621s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:31:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.755692792s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X..................X.................................] duration: 5.771251549s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:30:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.771276425s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................X..........XX...] duration: 18.460067217s trace: https://jaeger.dev.storj.io/trace/a0eab84190ca8f8 FAIL (2026-01-09 10:30:53 UTC) orbiter warning: context deadline exceeded Duration: 18.460092975s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 50 out of 60 is failed (threshold=6) [XXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX.XXXXXX] duration: 5.000931644s trace: https://jaeger.dev.storj.io/trace/61efe1d442efe60 FAIL (2026-01-09 10:30:49 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.0009505s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X..................X................................] duration: 5.811624241s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:30:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.811644309s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X..........................X................................] duration: 16.766294973s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:30:20 UTC) orbiter warning: context deadline exceeded Duration: 16.76630971s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X..........................X...............................] duration: 36.039406584s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:30:03 UTC) orbiter warning: context deadline exceeded Duration: 36.039436651s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................X..........XX..] duration: 17.491811747s trace: https://jaeger.dev.storj.io/trace/a0eab84190ca8f8 FAIL (2026-01-09 10:29:44 UTC) orbiter warning: context deadline exceeded Duration: 17.491861603s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 50 out of 60 is failed (threshold=6) [XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX.XXXXX] duration: 5.000617684s trace: https://jaeger.dev.storj.io/trace/58cb3c0577e60400 FAIL (2026-01-09 10:29:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000648673s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X..........................X..............................] duration: 12.805341032s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:29:19 UTC) orbiter warning: context deadline exceeded Duration: 12.805368664s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X..................X...............................] duration: 5.83662021s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:29:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.836658644s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................X..........XX.] duration: 30.001736964s trace: https://jaeger.dev.storj.io/trace/a0eab84190ca8f8 FAIL (2026-01-09 10:29:11 UTC) orbiter warning: context deadline exceeded Duration: 30.001762022s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X..................X..............................] duration: 5.840143817s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:28:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.840243848s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [.XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX.XXXX] duration: 5.002287549s trace: https://jaeger.dev.storj.io/trace/34918faca4bf23da FAIL (2026-01-09 10:28:41 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002307237s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................X..........XX] duration: 30.000656746s trace: https://jaeger.dev.storj.io/trace/a0eab84190ca8f8 FAIL (2026-01-09 10:28:27 UTC) orbiter warning: context deadline exceeded Duration: 30.000669842s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...X..........................X.............................] duration: 13.606467612s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:28:15 UTC) orbiter warning: context deadline exceeded Duration: 13.606485576s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [X.XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX.XXX] duration: 5.000908667s trace: https://jaeger.dev.storj.io/trace/786e61bc3cf982a1 FAIL (2026-01-09 10:28:02 UTC) orbiter warning: context deadline exceeded Duration: 5.000937991s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X..........X] duration: 30.001113133s trace: https://jaeger.dev.storj.io/trace/5e7e03757cbd6b10 FAIL (2026-01-09 10:27:57 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001133873s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X..................X.............................] duration: 5.788542397s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:27:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.788572235s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....X..........................X............................] duration: 20.11300985s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:27:42 UTC) orbiter warning: context deadline exceeded Duration: 20.113036351s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [XX.XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX.XX] duration: 5.000671716s trace: https://jaeger.dev.storj.io/trace/32b21e89a69fc72b FAIL (2026-01-09 10:27:14 UTC) orbiter warning: context deadline exceeded Duration: 5.000684662s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X..................X............................] duration: 5.787248117s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:27:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.787279817s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 22.63556379s trace: https://jaeger.dev.storj.io/trace/7ab59755fbcbaecc FAIL (2026-01-09 10:26:56 UTC) orbiter warning: context deadline exceeded Duration: 22.635583148s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [.XX.XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX.X] duration: 5.000772897s trace: https://jaeger.dev.storj.io/trace/29e901a6b156e46a FAIL (2026-01-09 10:26:37 UTC) orbiter warning: context deadline exceeded Duration: 5.000793296s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X............X..................X...........................] duration: 5.882701858s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:26:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.882728999s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....X..........................X...........................] duration: 16.348495349s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:26:32 UTC) orbiter warning: context deadline exceeded Duration: 16.348522652s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X............X..................X..........................] duration: 5.840114275s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:25:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.840139334s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 18.425469062s trace: https://jaeger.dev.storj.io/trace/7ab59755fbcbaecc FAIL (2026-01-09 10:25:59 UTC) orbiter warning: context deadline exceeded Duration: 18.425490923s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......X..........................X..........................] duration: 26.415181486s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:25:31 UTC) orbiter warning: context deadline exceeded Duration: 26.415390964s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [X.XX.XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX.] duration: 3.510411504s trace: https://jaeger.dev.storj.io/trace/21c8f0155cfae1e FAIL (2026-01-09 10:25:21 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.51043608s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X............X..................X.........................] duration: 5.854333461s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:25:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.854359761s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 27.4088809s trace: https://jaeger.dev.storj.io/trace/7ab59755fbcbaecc FAIL (2026-01-09 10:25:04 UTC) orbiter warning: context deadline exceeded Duration: 27.408908142s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......X..........................X.........................] duration: 19.156559983s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:25:02 UTC) orbiter warning: context deadline exceeded Duration: 19.15673202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X............X..................X........................] duration: 5.723103484s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:24:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.723130957s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 27.923514264s trace: https://jaeger.dev.storj.io/trace/7ab59755fbcbaecc FAIL (2026-01-09 10:24:36 UTC) orbiter warning: context deadline exceeded Duration: 27.923540634s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [XX.XX.XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXXX] duration: 5.001796426s trace: https://jaeger.dev.storj.io/trace/21c8f0155cfae1e FAIL (2026-01-09 10:24:29 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001820783s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X............X..................X.......................] duration: 5.799233903s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:23:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.799263258s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........X..........................X........................] duration: 21.742114734s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:23:53 UTC) orbiter warning: context deadline exceeded Duration: 21.742138008s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 20.363489512s trace: https://jaeger.dev.storj.io/trace/7ab59755fbcbaecc FAIL (2026-01-09 10:23:50 UTC) orbiter warning: context deadline exceeded Duration: 20.363512497s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 22.252029084s trace: https://jaeger.dev.storj.io/trace/7ab59755fbcbaecc FAIL (2026-01-09 10:23:30 UTC) orbiter warning: context deadline exceeded Duration: 22.252052959s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [XXX.XX.XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXXX] duration: 5.001042764s trace: https://jaeger.dev.storj.io/trace/3509bcfeeaf37695 FAIL (2026-01-09 10:23:27 UTC) orbiter warning: context deadline exceeded Duration: 5.001070737s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [XXXX.XX.XXXX.XXXX.X.X..XXXX.XXXXXXX.XXXXXXXXXXX.XXXXX.XXXXXX] duration: 5.001319343s trace: https://jaeger.dev.storj.io/trace/2e5224578bfa6239 FAIL (2026-01-09 10:23:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001345273s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........X..........................X.......................] duration: 6.012554682s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:23:06 UTC) orbiter warning: context deadline exceeded Duration: 6.012582705s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X............X..................X......................] duration: 5.955650369s trace: https://jaeger.dev.storj.io/trace/7989594c928c2c47 FAIL (2026-01-09 10:23:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 9 10:08:24.999399203 (834h39m24.870482238s) [upload=730081]: Starting upload Jan 9 10:08:24.999454759 (834h39m24.870537794s) [upload=730081]: Done starting upload Jan 9 10:08:24.999587080 (834h39m24.870670115s) [upload=730081]: Uploading segments... Jan 9 10:08:24.999682833 (834h39m24.870765868s) [upload=730081]: Got next segment. Inline: false Jan 9 10:08:24.999730904 (834h39m24.870813969s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 10:08:24.999739841 (834h39m24.870822877s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 10:08:25.244623117 (834h39m25.115706162s) [seg_pos={0 0}, upload=730081]: Begin upload segment... Jan 9 10:08:25.244679764 (834h39m25.115762799s) [seg_pos={0 0}, upload=730081]: Done begin upload segment. Jan 9 10:08:25.244724179 (834h39m25.115807214s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.244733517 (834h39m25.115816552s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245143287 (834h39m25.116226322s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245471833 (834h39m25.116554868s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245611238 (834h39m25.116694273s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.245840094 (834h39m25.116923129s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246007663 (834h39m25.117090699s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246200270 (834h39m25.117283305s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246290091 (834h39m25.117373126s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246516422 (834h39m25.117599457s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246535769 (834h39m25.117618804s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246782398 (834h39m25.117865433s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.246827033 (834h39m25.117910068s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247032605 (834h39m25.11811564s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247292028 (834h39m25.118375063s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:25.247530022 (834h39m25.118613057s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Uploading piece... Jan 9 10:08:26.882456564 (834h39m26.753539599s) [upload=730081]: Next returned nil segment Jan 9 10:08:26.882481762 (834h39m26.753564797s) [upload=730081]: Waiting for error group managing segments... Jan 9 10:08:27.203993871 (834h39m27.075076906s) [noise=true , address=77.90.31.14:4662 , piece_id=J664OYIONM7FEKXEZTPRUZHYIZ5PCEF5WFMRF5NUQ3PMKMGVUXOQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.205327059 (834h39m27.076410094s) [noise=true , address=77.90.31.11:4555 , piece_id=QM465N6DSDZ6XYMC5DTTHC4VM7OHAAMU56Z25TYCGJUSWGR2Q27Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.210368088 (834h39m27.081451133s) [noise=true , address=77.90.31.19:4842 , piece_id=UKVOKHPJ6RD24ITJ42X4CD5Z3HYCH6IE66YD6LI4CQUOXDJBOCFA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.222911873 (834h39m27.093994918s) [noise=true , address=77.90.31.15:4697 , piece_id=YOLSRQQJZKX73PPFCAO46ZALKMPVXK3RLJ6ATBHRKPCPW2K5QEWA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.244114366 (834h39m27.115197402s) [noise=true , address=145.79.227.33:4869 , piece_id=TZFAOCDWAY23VAB67L2PLRH4RI7HTZCWKKJHLJTSF4GTYDT2O6OQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.249719249 (834h39m27.120802284s) [noise=true , address=145.79.227.49:4339 , piece_id=E35KY7OAU2FZZHHU5J57HHJGMOE22OWO57JTH3P5RSQCK24G6CVA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.251625378 (834h39m27.122708423s) [noise=true , address=145.79.227.39:4013 , piece_id=V7ZTSHUECC2EWLZSZJK3BJNGQ7HW72YK7SIMNA7KNVSWKW3DRVGQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.258573469 (834h39m27.129656514s) [noise=true , address=145.79.227.18:4007 , piece_id=2KF6RJNEQAA3KKXOTVEAMPIVNJ4HYLJUJ6B42ZADO6JP3QGKNVBA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.259785627 (834h39m27.130868662s) [noise=true , address=145.79.227.16:4912 , piece_id=P727EI4R6ST42KEXRSCOEBKAVGPZO5OGCAQMD26IAVDML5P5GJ3Q, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.262470129 (834h39m27.133553174s) [noise=true , address=145.79.227.45:4230 , piece_id=A4VX2KSYFAMND4OQEWDBKBNEFB34KPDPAEKGHCSH6M25PTAF2E5A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.329451572 (834h39m27.200534607s) [noise=true , address=145.79.227.15:4876 , piece_id=3SJUOXVH6DVC5W4LHI5AEECZNXRIGGOTCOHAGI55BUW3XVEFNDVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.332881472 (834h39m27.203964507s) [noise=true , address=145.79.227.52:4445 , piece_id=KGOM66JUJTOWCPRVT3X2NT6HKBMBXONV5YVGSDEDSTTOIZSVN2GQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338435529 (834h39m27.209518574s) [noise=true , address=145.79.227.41:4087 , piece_id=HDCTSIKC4FIDALNSFWI7C27RELK62VHC7EI3WDYJJHV5GUNSAO3A, seg_pos={0 0}, upload=730081]: Done uploading piece. err: <nil> Jan 9 10:08:27.338452731 (834h39m27.209535766s) [seg_pos={0 0}, upload=730081]: Segment reached optimal threshold of 13 pieces. Jan 9 10:08:27.338679143 (834h39m27.209762178s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338689322 (834h39m27.209772357s) [noise=true , address=145.79.227.5:4221 , piece_id=YGHDYB4J5ORKOM3JL5HZFHEACCKN5M63VN4YEZZPC5LLB2MF63HQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.338833366 (834h39m27.209916401s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.338850639 (834h39m27.209933684s) [noise=true , address=45.91.116.5:4298 , piece_id=XJN4DHSXOQ4FMSUFFTOECFBEPXVDMAZHY3GQCDRJWDFTBZSV4BVQ, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339026875 (834h39m27.21010991s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Jan 9 10:08:27.339036413 (834h39m27.210119468s) [noise=true , address=45.91.116.6:4975 , piece_id=HKAOQFFOLDFLQPYSM4H4EETYW3AVQ4WDEA3QWIHMXDERAB5EYFIA, seg_pos={0 0}, upload=730081]: Long tail cancellation due to optimal threshold Jan 9 10:08:27.339057292 (834h39m27.210140327s) [seg_pos={0 0}, upload=730081]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 10:08:27.339140331 (834h39m27.210223366s) [upload=730081]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 10:08:27.339146091 (834h39m27.210229126s) [upload=730081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 10:08:27.632776159 (834h39m27.503859194s) [upload=730081]: Done uploading segments... Jan 9 10:08:27.632794182 (834h39m27.503877218s) [upload=730081]: Upload finished. err: <nil> Duration: 5.955679934s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 27.056797988s trace: https://jaeger.dev.storj.io/trace/7ab59755fbcbaecc FAIL (2026-01-09 10:22:58 UTC) orbiter warning: context deadline exceeded Duration: 27.05681449s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 21.904478919s trace: https://jaeger.dev.storj.io/trace/7ab59755fbcbaecc FAIL (2026-01-09 10:22:30 UTC) orbiter warning: context deadline exceeded Duration: 21.905028237s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........X..........................X......................] duration: 17.450063608s trace: https://jaeger.dev.storj.io/trace/34583af1656b6c58 FAIL (2026-01-09 10:22:16 UTC) orbiter warning: context deadline exceeded Duration: 17.450089659s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)