Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [...X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...X.......] duration: 7.041455363s trace: https://jaeger.dev.storj.io/trace/6757f0cde8f7cce4 FAIL (2025-12-26 23:57:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:51:20.153781627 (512h22m20.024864672s) [upload=449601]: Starting upload Dec 26 23:51:20.153854327 (512h22m20.024937362s) [upload=449601]: Done starting upload Dec 26 23:51:20.153991797 (512h22m20.025074842s) [upload=449601]: Uploading segments... Dec 26 23:51:20.154215294 (512h22m20.025298329s) [upload=449601]: Got next segment. Inline: false Dec 26 23:51:20.154261251 (512h22m20.025344286s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:51:20.154269416 (512h22m20.025352451s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:51:20.403820138 (512h22m20.274903183s) [seg_pos={0 0}, upload=449601]: Begin upload segment... Dec 26 23:51:20.403872798 (512h22m20.274955833s) [seg_pos={0 0}, upload=449601]: Done begin upload segment. Dec 26 23:51:20.403906322 (512h22m20.274989357s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.403956037 (512h22m20.275039072s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404523297 (512h22m20.275606332s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404934159 (512h22m20.276017194s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405323210 (512h22m20.276406255s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405784388 (512h22m20.276867423s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.406353832 (512h22m20.277436877s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407445070 (512h22m20.278528105s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407702831 (512h22m20.278785866s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.408618514 (512h22m20.279701549s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409314389 (512h22m20.280397424s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409970459 (512h22m20.281053504s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.411235437 (512h22m20.282318472s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.412646294 (512h22m20.283729329s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.413935268 (512h22m20.285018303s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.416127852 (512h22m20.287210897s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:22.123323865 (512h22m21.9944069s) [upload=449601]: Next returned nil segment Dec 26 23:51:22.123348112 (512h22m21.994431147s) [upload=449601]: Waiting for error group managing segments... Dec 26 23:51:22.409544009 (512h22m22.280627054s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.413146688 (512h22m22.284229723s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.415595853 (512h22m22.286678898s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.427233401 (512h22m22.298316436s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432330937 (512h22m22.303413972s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432667158 (512h22m22.303750193s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.470249649 (512h22m22.341332684s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.471883119 (512h22m22.342966154s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.626854321 (512h22m22.497937367s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.649898530 (512h22m22.520981565s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.651253370 (512h22m22.522336415s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.658475341 (512h22m22.529558376s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699461663 (512h22m22.570544698s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699488233 (512h22m22.570571269s) [seg_pos={0 0}, upload=449601]: Segment reached optimal threshold of 13 pieces. Dec 26 23:51:22.699772424 (512h22m22.57085546s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699782394 (512h22m22.570865429s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.699933992 (512h22m22.571017028s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699953359 (512h22m22.571036394s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700197413 (512h22m22.571280449s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: read tcp 188.166.225.170:35306->77.90.31.13:4627: use of closed network connection; context canceled Dec 26 23:51:22.700203997 (512h22m22.571287032s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700232180 (512h22m22.571315215s) [seg_pos={0 0}, upload=449601]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:51:22.700318274 (512h22m22.571401309s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:51:22.700322793 (512h22m22.571405828s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:51:22.994514616 (512h22m22.865597661s) [upload=449601]: Done uploading segments... Dec 26 23:51:22.994530666 (512h22m22.865613701s) [upload=449601]: Upload finished. err: <nil> Duration: 7.041479289s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 237.195596ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:56:49 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 237.224771ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [....X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...X......] duration: 6.128195115s trace: https://jaeger.dev.storj.io/trace/6757f0cde8f7cce4 FAIL (2025-12-26 23:56:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:51:20.153781627 (512h22m20.024864672s) [upload=449601]: Starting upload Dec 26 23:51:20.153854327 (512h22m20.024937362s) [upload=449601]: Done starting upload Dec 26 23:51:20.153991797 (512h22m20.025074842s) [upload=449601]: Uploading segments... Dec 26 23:51:20.154215294 (512h22m20.025298329s) [upload=449601]: Got next segment. Inline: false Dec 26 23:51:20.154261251 (512h22m20.025344286s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:51:20.154269416 (512h22m20.025352451s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:51:20.403820138 (512h22m20.274903183s) [seg_pos={0 0}, upload=449601]: Begin upload segment... Dec 26 23:51:20.403872798 (512h22m20.274955833s) [seg_pos={0 0}, upload=449601]: Done begin upload segment. Dec 26 23:51:20.403906322 (512h22m20.274989357s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.403956037 (512h22m20.275039072s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404523297 (512h22m20.275606332s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404934159 (512h22m20.276017194s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405323210 (512h22m20.276406255s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405784388 (512h22m20.276867423s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.406353832 (512h22m20.277436877s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407445070 (512h22m20.278528105s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407702831 (512h22m20.278785866s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.408618514 (512h22m20.279701549s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409314389 (512h22m20.280397424s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409970459 (512h22m20.281053504s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.411235437 (512h22m20.282318472s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.412646294 (512h22m20.283729329s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.413935268 (512h22m20.285018303s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.416127852 (512h22m20.287210897s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:22.123323865 (512h22m21.9944069s) [upload=449601]: Next returned nil segment Dec 26 23:51:22.123348112 (512h22m21.994431147s) [upload=449601]: Waiting for error group managing segments... Dec 26 23:51:22.409544009 (512h22m22.280627054s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.413146688 (512h22m22.284229723s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.415595853 (512h22m22.286678898s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.427233401 (512h22m22.298316436s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432330937 (512h22m22.303413972s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432667158 (512h22m22.303750193s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.470249649 (512h22m22.341332684s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.471883119 (512h22m22.342966154s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.626854321 (512h22m22.497937367s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.649898530 (512h22m22.520981565s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.651253370 (512h22m22.522336415s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.658475341 (512h22m22.529558376s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699461663 (512h22m22.570544698s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699488233 (512h22m22.570571269s) [seg_pos={0 0}, upload=449601]: Segment reached optimal threshold of 13 pieces. Dec 26 23:51:22.699772424 (512h22m22.57085546s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699782394 (512h22m22.570865429s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.699933992 (512h22m22.571017028s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699953359 (512h22m22.571036394s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700197413 (512h22m22.571280449s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: read tcp 188.166.225.170:35306->77.90.31.13:4627: use of closed network connection; context canceled Dec 26 23:51:22.700203997 (512h22m22.571287032s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700232180 (512h22m22.571315215s) [seg_pos={0 0}, upload=449601]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:51:22.700318274 (512h22m22.571401309s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:51:22.700322793 (512h22m22.571405828s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:51:22.994514616 (512h22m22.865597661s) [upload=449601]: Done uploading segments... Dec 26 23:51:22.994530666 (512h22m22.865613701s) [upload=449601]: Upload finished. err: <nil> Duration: 6.128222306s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.....X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...X.....] duration: 6.031215647s trace: https://jaeger.dev.storj.io/trace/6757f0cde8f7cce4 FAIL (2025-12-26 23:55:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:51:20.153781627 (512h22m20.024864672s) [upload=449601]: Starting upload Dec 26 23:51:20.153854327 (512h22m20.024937362s) [upload=449601]: Done starting upload Dec 26 23:51:20.153991797 (512h22m20.025074842s) [upload=449601]: Uploading segments... Dec 26 23:51:20.154215294 (512h22m20.025298329s) [upload=449601]: Got next segment. Inline: false Dec 26 23:51:20.154261251 (512h22m20.025344286s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:51:20.154269416 (512h22m20.025352451s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:51:20.403820138 (512h22m20.274903183s) [seg_pos={0 0}, upload=449601]: Begin upload segment... Dec 26 23:51:20.403872798 (512h22m20.274955833s) [seg_pos={0 0}, upload=449601]: Done begin upload segment. Dec 26 23:51:20.403906322 (512h22m20.274989357s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.403956037 (512h22m20.275039072s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404523297 (512h22m20.275606332s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404934159 (512h22m20.276017194s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405323210 (512h22m20.276406255s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405784388 (512h22m20.276867423s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.406353832 (512h22m20.277436877s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407445070 (512h22m20.278528105s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407702831 (512h22m20.278785866s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.408618514 (512h22m20.279701549s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409314389 (512h22m20.280397424s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409970459 (512h22m20.281053504s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.411235437 (512h22m20.282318472s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.412646294 (512h22m20.283729329s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.413935268 (512h22m20.285018303s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.416127852 (512h22m20.287210897s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:22.123323865 (512h22m21.9944069s) [upload=449601]: Next returned nil segment Dec 26 23:51:22.123348112 (512h22m21.994431147s) [upload=449601]: Waiting for error group managing segments... Dec 26 23:51:22.409544009 (512h22m22.280627054s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.413146688 (512h22m22.284229723s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.415595853 (512h22m22.286678898s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.427233401 (512h22m22.298316436s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432330937 (512h22m22.303413972s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432667158 (512h22m22.303750193s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.470249649 (512h22m22.341332684s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.471883119 (512h22m22.342966154s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.626854321 (512h22m22.497937367s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.649898530 (512h22m22.520981565s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.651253370 (512h22m22.522336415s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.658475341 (512h22m22.529558376s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699461663 (512h22m22.570544698s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699488233 (512h22m22.570571269s) [seg_pos={0 0}, upload=449601]: Segment reached optimal threshold of 13 pieces. Dec 26 23:51:22.699772424 (512h22m22.57085546s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699782394 (512h22m22.570865429s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.699933992 (512h22m22.571017028s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699953359 (512h22m22.571036394s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700197413 (512h22m22.571280449s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: read tcp 188.166.225.170:35306->77.90.31.13:4627: use of closed network connection; context canceled Dec 26 23:51:22.700203997 (512h22m22.571287032s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700232180 (512h22m22.571315215s) [seg_pos={0 0}, upload=449601]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:51:22.700318274 (512h22m22.571401309s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:51:22.700322793 (512h22m22.571405828s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:51:22.994514616 (512h22m22.865597661s) [upload=449601]: Done uploading segments... Dec 26 23:51:22.994530666 (512h22m22.865613701s) [upload=449601]: Upload finished. err: <nil> Duration: 6.031245605s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 256.058973ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:55:51 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 256.084342ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [......X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...X....] duration: 18.207296816s trace: https://jaeger.dev.storj.io/trace/6757f0cde8f7cce4 FAIL (2025-12-26 23:55:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:51:20.153781627 (512h22m20.024864672s) [upload=449601]: Starting upload Dec 26 23:51:20.153854327 (512h22m20.024937362s) [upload=449601]: Done starting upload Dec 26 23:51:20.153991797 (512h22m20.025074842s) [upload=449601]: Uploading segments... Dec 26 23:51:20.154215294 (512h22m20.025298329s) [upload=449601]: Got next segment. Inline: false Dec 26 23:51:20.154261251 (512h22m20.025344286s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:51:20.154269416 (512h22m20.025352451s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:51:20.403820138 (512h22m20.274903183s) [seg_pos={0 0}, upload=449601]: Begin upload segment... Dec 26 23:51:20.403872798 (512h22m20.274955833s) [seg_pos={0 0}, upload=449601]: Done begin upload segment. Dec 26 23:51:20.403906322 (512h22m20.274989357s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.403956037 (512h22m20.275039072s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404523297 (512h22m20.275606332s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404934159 (512h22m20.276017194s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405323210 (512h22m20.276406255s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405784388 (512h22m20.276867423s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.406353832 (512h22m20.277436877s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407445070 (512h22m20.278528105s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407702831 (512h22m20.278785866s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.408618514 (512h22m20.279701549s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409314389 (512h22m20.280397424s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409970459 (512h22m20.281053504s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.411235437 (512h22m20.282318472s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.412646294 (512h22m20.283729329s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.413935268 (512h22m20.285018303s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.416127852 (512h22m20.287210897s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:22.123323865 (512h22m21.9944069s) [upload=449601]: Next returned nil segment Dec 26 23:51:22.123348112 (512h22m21.994431147s) [upload=449601]: Waiting for error group managing segments... Dec 26 23:51:22.409544009 (512h22m22.280627054s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.413146688 (512h22m22.284229723s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.415595853 (512h22m22.286678898s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.427233401 (512h22m22.298316436s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432330937 (512h22m22.303413972s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432667158 (512h22m22.303750193s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.470249649 (512h22m22.341332684s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.471883119 (512h22m22.342966154s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.626854321 (512h22m22.497937367s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.649898530 (512h22m22.520981565s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.651253370 (512h22m22.522336415s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.658475341 (512h22m22.529558376s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699461663 (512h22m22.570544698s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699488233 (512h22m22.570571269s) [seg_pos={0 0}, upload=449601]: Segment reached optimal threshold of 13 pieces. Dec 26 23:51:22.699772424 (512h22m22.57085546s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699782394 (512h22m22.570865429s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.699933992 (512h22m22.571017028s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699953359 (512h22m22.571036394s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700197413 (512h22m22.571280449s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: read tcp 188.166.225.170:35306->77.90.31.13:4627: use of closed network connection; context canceled Dec 26 23:51:22.700203997 (512h22m22.571287032s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700232180 (512h22m22.571315215s) [seg_pos={0 0}, upload=449601]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:51:22.700318274 (512h22m22.571401309s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:51:22.700322793 (512h22m22.571405828s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:51:22.994514616 (512h22m22.865597661s) [upload=449601]: Done uploading segments... Dec 26 23:51:22.994530666 (512h22m22.865613701s) [upload=449601]: Upload finished. err: <nil> Duration: 18.207323308s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 237.00388ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:55:13 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 237.034318ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 244.041562ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:54:28 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 244.075777ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.......X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...X...] duration: 18.965175637s trace: https://jaeger.dev.storj.io/trace/6757f0cde8f7cce4 FAIL (2025-12-26 23:54:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:51:20.153781627 (512h22m20.024864672s) [upload=449601]: Starting upload Dec 26 23:51:20.153854327 (512h22m20.024937362s) [upload=449601]: Done starting upload Dec 26 23:51:20.153991797 (512h22m20.025074842s) [upload=449601]: Uploading segments... Dec 26 23:51:20.154215294 (512h22m20.025298329s) [upload=449601]: Got next segment. Inline: false Dec 26 23:51:20.154261251 (512h22m20.025344286s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:51:20.154269416 (512h22m20.025352451s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:51:20.403820138 (512h22m20.274903183s) [seg_pos={0 0}, upload=449601]: Begin upload segment... Dec 26 23:51:20.403872798 (512h22m20.274955833s) [seg_pos={0 0}, upload=449601]: Done begin upload segment. Dec 26 23:51:20.403906322 (512h22m20.274989357s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.403956037 (512h22m20.275039072s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404523297 (512h22m20.275606332s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404934159 (512h22m20.276017194s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405323210 (512h22m20.276406255s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405784388 (512h22m20.276867423s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.406353832 (512h22m20.277436877s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407445070 (512h22m20.278528105s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407702831 (512h22m20.278785866s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.408618514 (512h22m20.279701549s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409314389 (512h22m20.280397424s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409970459 (512h22m20.281053504s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.411235437 (512h22m20.282318472s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.412646294 (512h22m20.283729329s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.413935268 (512h22m20.285018303s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.416127852 (512h22m20.287210897s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:22.123323865 (512h22m21.9944069s) [upload=449601]: Next returned nil segment Dec 26 23:51:22.123348112 (512h22m21.994431147s) [upload=449601]: Waiting for error group managing segments... Dec 26 23:51:22.409544009 (512h22m22.280627054s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.413146688 (512h22m22.284229723s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.415595853 (512h22m22.286678898s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.427233401 (512h22m22.298316436s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432330937 (512h22m22.303413972s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432667158 (512h22m22.303750193s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.470249649 (512h22m22.341332684s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.471883119 (512h22m22.342966154s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.626854321 (512h22m22.497937367s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.649898530 (512h22m22.520981565s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.651253370 (512h22m22.522336415s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.658475341 (512h22m22.529558376s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699461663 (512h22m22.570544698s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699488233 (512h22m22.570571269s) [seg_pos={0 0}, upload=449601]: Segment reached optimal threshold of 13 pieces. Dec 26 23:51:22.699772424 (512h22m22.57085546s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699782394 (512h22m22.570865429s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.699933992 (512h22m22.571017028s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699953359 (512h22m22.571036394s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700197413 (512h22m22.571280449s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: read tcp 188.166.225.170:35306->77.90.31.13:4627: use of closed network connection; context canceled Dec 26 23:51:22.700203997 (512h22m22.571287032s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700232180 (512h22m22.571315215s) [seg_pos={0 0}, upload=449601]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:51:22.700318274 (512h22m22.571401309s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:51:22.700322793 (512h22m22.571405828s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:51:22.994514616 (512h22m22.865597661s) [upload=449601]: Done uploading segments... Dec 26 23:51:22.994530666 (512h22m22.865613701s) [upload=449601]: Upload finished. err: <nil> Duration: 18.965204402s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 243.670635ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:54:07 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 243.703196ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [........X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...X..] duration: 18.646056486s trace: https://jaeger.dev.storj.io/trace/6757f0cde8f7cce4 FAIL (2025-12-26 23:53:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:51:20.153781627 (512h22m20.024864672s) [upload=449601]: Starting upload Dec 26 23:51:20.153854327 (512h22m20.024937362s) [upload=449601]: Done starting upload Dec 26 23:51:20.153991797 (512h22m20.025074842s) [upload=449601]: Uploading segments... Dec 26 23:51:20.154215294 (512h22m20.025298329s) [upload=449601]: Got next segment. Inline: false Dec 26 23:51:20.154261251 (512h22m20.025344286s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:51:20.154269416 (512h22m20.025352451s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:51:20.403820138 (512h22m20.274903183s) [seg_pos={0 0}, upload=449601]: Begin upload segment... Dec 26 23:51:20.403872798 (512h22m20.274955833s) [seg_pos={0 0}, upload=449601]: Done begin upload segment. Dec 26 23:51:20.403906322 (512h22m20.274989357s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.403956037 (512h22m20.275039072s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404523297 (512h22m20.275606332s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404934159 (512h22m20.276017194s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405323210 (512h22m20.276406255s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405784388 (512h22m20.276867423s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.406353832 (512h22m20.277436877s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407445070 (512h22m20.278528105s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407702831 (512h22m20.278785866s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.408618514 (512h22m20.279701549s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409314389 (512h22m20.280397424s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409970459 (512h22m20.281053504s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.411235437 (512h22m20.282318472s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.412646294 (512h22m20.283729329s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.413935268 (512h22m20.285018303s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.416127852 (512h22m20.287210897s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:22.123323865 (512h22m21.9944069s) [upload=449601]: Next returned nil segment Dec 26 23:51:22.123348112 (512h22m21.994431147s) [upload=449601]: Waiting for error group managing segments... Dec 26 23:51:22.409544009 (512h22m22.280627054s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.413146688 (512h22m22.284229723s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.415595853 (512h22m22.286678898s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.427233401 (512h22m22.298316436s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432330937 (512h22m22.303413972s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432667158 (512h22m22.303750193s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.470249649 (512h22m22.341332684s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.471883119 (512h22m22.342966154s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.626854321 (512h22m22.497937367s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.649898530 (512h22m22.520981565s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.651253370 (512h22m22.522336415s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.658475341 (512h22m22.529558376s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699461663 (512h22m22.570544698s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699488233 (512h22m22.570571269s) [seg_pos={0 0}, upload=449601]: Segment reached optimal threshold of 13 pieces. Dec 26 23:51:22.699772424 (512h22m22.57085546s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699782394 (512h22m22.570865429s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.699933992 (512h22m22.571017028s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699953359 (512h22m22.571036394s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700197413 (512h22m22.571280449s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: read tcp 188.166.225.170:35306->77.90.31.13:4627: use of closed network connection; context canceled Dec 26 23:51:22.700203997 (512h22m22.571287032s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700232180 (512h22m22.571315215s) [seg_pos={0 0}, upload=449601]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:51:22.700318274 (512h22m22.571401309s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:51:22.700322793 (512h22m22.571405828s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:51:22.994514616 (512h22m22.865597661s) [upload=449601]: Done uploading segments... Dec 26 23:51:22.994530666 (512h22m22.865613701s) [upload=449601]: Upload finished. err: <nil> Duration: 18.646083929s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 246.964718ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:53:33 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 246.980888ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 245.78393ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:53:03 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 245.818676ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.........X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...X.] duration: 6.0528707s trace: https://jaeger.dev.storj.io/trace/6757f0cde8f7cce4 FAIL (2025-12-26 23:52:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:51:20.153781627 (512h22m20.024864672s) [upload=449601]: Starting upload Dec 26 23:51:20.153854327 (512h22m20.024937362s) [upload=449601]: Done starting upload Dec 26 23:51:20.153991797 (512h22m20.025074842s) [upload=449601]: Uploading segments... Dec 26 23:51:20.154215294 (512h22m20.025298329s) [upload=449601]: Got next segment. Inline: false Dec 26 23:51:20.154261251 (512h22m20.025344286s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:51:20.154269416 (512h22m20.025352451s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:51:20.403820138 (512h22m20.274903183s) [seg_pos={0 0}, upload=449601]: Begin upload segment... Dec 26 23:51:20.403872798 (512h22m20.274955833s) [seg_pos={0 0}, upload=449601]: Done begin upload segment. Dec 26 23:51:20.403906322 (512h22m20.274989357s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.403956037 (512h22m20.275039072s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404523297 (512h22m20.275606332s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404934159 (512h22m20.276017194s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405323210 (512h22m20.276406255s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405784388 (512h22m20.276867423s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.406353832 (512h22m20.277436877s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407445070 (512h22m20.278528105s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407702831 (512h22m20.278785866s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.408618514 (512h22m20.279701549s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409314389 (512h22m20.280397424s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409970459 (512h22m20.281053504s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.411235437 (512h22m20.282318472s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.412646294 (512h22m20.283729329s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.413935268 (512h22m20.285018303s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.416127852 (512h22m20.287210897s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:22.123323865 (512h22m21.9944069s) [upload=449601]: Next returned nil segment Dec 26 23:51:22.123348112 (512h22m21.994431147s) [upload=449601]: Waiting for error group managing segments... Dec 26 23:51:22.409544009 (512h22m22.280627054s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.413146688 (512h22m22.284229723s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.415595853 (512h22m22.286678898s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.427233401 (512h22m22.298316436s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432330937 (512h22m22.303413972s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432667158 (512h22m22.303750193s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.470249649 (512h22m22.341332684s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.471883119 (512h22m22.342966154s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.626854321 (512h22m22.497937367s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.649898530 (512h22m22.520981565s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.651253370 (512h22m22.522336415s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.658475341 (512h22m22.529558376s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699461663 (512h22m22.570544698s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699488233 (512h22m22.570571269s) [seg_pos={0 0}, upload=449601]: Segment reached optimal threshold of 13 pieces. Dec 26 23:51:22.699772424 (512h22m22.57085546s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699782394 (512h22m22.570865429s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.699933992 (512h22m22.571017028s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699953359 (512h22m22.571036394s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700197413 (512h22m22.571280449s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: read tcp 188.166.225.170:35306->77.90.31.13:4627: use of closed network connection; context canceled Dec 26 23:51:22.700203997 (512h22m22.571287032s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700232180 (512h22m22.571315215s) [seg_pos={0 0}, upload=449601]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:51:22.700318274 (512h22m22.571401309s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:51:22.700322793 (512h22m22.571405828s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:51:22.994514616 (512h22m22.865597661s) [upload=449601]: Done uploading segments... Dec 26 23:51:22.994530666 (512h22m22.865613701s) [upload=449601]: Upload finished. err: <nil> Duration: 6.052900427s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 250.733834ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:51:58 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 250.76822ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..........X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...X] duration: 30.002494324s trace: https://jaeger.dev.storj.io/trace/6757f0cde8f7cce4 FAIL (2025-12-26 23:51:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:51:20.153781627 (512h22m20.024864672s) [upload=449601]: Starting upload Dec 26 23:51:20.153854327 (512h22m20.024937362s) [upload=449601]: Done starting upload Dec 26 23:51:20.153991797 (512h22m20.025074842s) [upload=449601]: Uploading segments... Dec 26 23:51:20.154215294 (512h22m20.025298329s) [upload=449601]: Got next segment. Inline: false Dec 26 23:51:20.154261251 (512h22m20.025344286s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:51:20.154269416 (512h22m20.025352451s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:51:20.403820138 (512h22m20.274903183s) [seg_pos={0 0}, upload=449601]: Begin upload segment... Dec 26 23:51:20.403872798 (512h22m20.274955833s) [seg_pos={0 0}, upload=449601]: Done begin upload segment. Dec 26 23:51:20.403906322 (512h22m20.274989357s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.403956037 (512h22m20.275039072s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404523297 (512h22m20.275606332s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.404934159 (512h22m20.276017194s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405323210 (512h22m20.276406255s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.405784388 (512h22m20.276867423s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.406353832 (512h22m20.277436877s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407445070 (512h22m20.278528105s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.407702831 (512h22m20.278785866s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.408618514 (512h22m20.279701549s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409314389 (512h22m20.280397424s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.409970459 (512h22m20.281053504s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.411235437 (512h22m20.282318472s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.412646294 (512h22m20.283729329s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.413935268 (512h22m20.285018303s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:20.416127852 (512h22m20.287210897s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Uploading piece... Dec 26 23:51:22.123323865 (512h22m21.9944069s) [upload=449601]: Next returned nil segment Dec 26 23:51:22.123348112 (512h22m21.994431147s) [upload=449601]: Waiting for error group managing segments... Dec 26 23:51:22.409544009 (512h22m22.280627054s) [noise=true , address=145.79.227.36:4517 , piece_id=I462OWJVZ7TIXMCJXS23ZRWKJIWNKIFVOCS7CELOYCAH4LS3M7YA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.413146688 (512h22m22.284229723s) [noise=true , address=145.79.227.15:4873 , piece_id=ZM25KXPKA46E2Z5P2GDRAHWXR7VI2TFURLLXOIV3WZT3RORSD26Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.415595853 (512h22m22.286678898s) [noise=true , address=145.79.227.37:4565 , piece_id=PYMC52TANG3YM4GGCLLSCRCBTNRHTHRYP5UXLKV53DRND24PNMDA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.427233401 (512h22m22.298316436s) [noise=true , address=145.79.227.39:4013 , piece_id=AVDVURSJKQAH3S2RII3DZPK765EZGTW6JUZCRWLZFMQZEPA73DVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432330937 (512h22m22.303413972s) [noise=true , address=145.79.227.6:4257 , piece_id=N4PXFEFMENWOU3OQWLYA2X47UDXBH564WRIMMBWSTEMRWEE4PCKQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.432667158 (512h22m22.303750193s) [noise=true , address=145.79.227.11:4731 , piece_id=WDMMVDO7PH6U3K6L7H7GMYGFCLHA5D5QADUPJB2H6VRSDS7XKWVA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.470249649 (512h22m22.341332684s) [noise=true , address=145.79.227.52:4446 , piece_id=WL4ANRH5NYGCBBHAX7RSEATCFHFE6F6PIF33B36ZIJATKAIAPDSA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.471883119 (512h22m22.342966154s) [noise=true , address=145.79.227.17:4971 , piece_id=BCYP3ZXMO5APPJTZXZFOKYQPCOAGFENIA467WJ2Y3OKDQNXSKZ5Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.626854321 (512h22m22.497937367s) [noise=true , address=77.90.31.17:4773 , piece_id=KQ7YO5NFFKRFUMTQOLPGTO7MZAZV2A4IFJCUTCGPXH7IGR7D2MXQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.649898530 (512h22m22.520981565s) [noise=true , address=77.90.31.12:4590 , piece_id=SZZ6DV4BV5TLFQ4XSWYUBEEFJTKTMUA4LOOHVPS52WBZ5CGSTMQA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.651253370 (512h22m22.522336415s) [noise=true , address=45.91.116.9:4263 , piece_id=72HIJRY4UWR2E2CLYEX5QI5FIW64WSDWYX64RSENZUOAY46XDQGA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.658475341 (512h22m22.529558376s) [noise=true , address=77.90.31.14:4662 , piece_id=NIJ4SF2VZMH2IHDOIEFUOAZKII5TEKY7C4YGCSLJP5VMQ2SRMMFA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699461663 (512h22m22.570544698s) [noise=true , address=77.90.31.11:4555 , piece_id=3JI2Q22CZFTJEEJ6FHR374MXVGWDKC2CAFPKZ5AQ4WJ5K574ZZUQ, seg_pos={0 0}, upload=449601]: Done uploading piece. err: <nil> Dec 26 23:51:22.699488233 (512h22m22.570571269s) [seg_pos={0 0}, upload=449601]: Segment reached optimal threshold of 13 pieces. Dec 26 23:51:22.699772424 (512h22m22.57085546s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699782394 (512h22m22.570865429s) [noise=true , address=45.91.116.8:4047 , piece_id=EENZU4I4WYV5KTOXNUG4BUOC5R4R6S5AWY7B2VS4PG2TRYN2SKYA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.699933992 (512h22m22.571017028s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy): protocol: expected piece hash; context canceled; EOF Dec 26 23:51:22.699953359 (512h22m22.571036394s) [noise=true , address=45.91.116.13:4156 , piece_id=7JD2FFXSQJC2EQ7BWPWMSZJ5CSVXNQ42IB27AATPBT6NJNZ7GA7Q, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700197413 (512h22m22.571280449s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: read tcp 188.166.225.170:35306->77.90.31.13:4627: use of closed network connection; context canceled Dec 26 23:51:22.700203997 (512h22m22.571287032s) [noise=true , address=77.90.31.13:4627 , piece_id=RHIEEMHIRR74BTHJYBKNR6DC3XKV7WYFWHTHUNFITPMGJ6SZ2ZIA, seg_pos={0 0}, upload=449601]: Long tail cancellation due to optimal threshold Dec 26 23:51:22.700232180 (512h22m22.571315215s) [seg_pos={0 0}, upload=449601]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:51:22.700318274 (512h22m22.571401309s) [upload=449601]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:51:22.700322793 (512h22m22.571405828s) [upload=449601]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:51:22.994514616 (512h22m22.865597661s) [upload=449601]: Done uploading segments... Dec 26 23:51:22.994530666 (512h22m22.865613701s) [upload=449601]: Upload finished. err: <nil> Duration: 30.002510715s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 244.185656ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:51:30 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 244.212859ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [...........X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX...] duration: 7.958843478s trace: https://jaeger.dev.storj.io/trace/105cfb5b0968a8f3 FAIL (2025-12-26 23:51:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA) context deadline exceeded; piecestore: (Node ID: 1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, Piece ID: UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A) context deadline exceeded ======LOGS====== Dec 26 23:49:08.013669624 (512h20m7.884752659s) [upload=449566]: Starting upload Dec 26 23:49:08.013722815 (512h20m7.88480585s) [upload=449566]: Done starting upload Dec 26 23:49:08.013923747 (512h20m7.885006792s) [upload=449566]: Uploading segments... Dec 26 23:49:08.014118418 (512h20m7.885201463s) [upload=449566]: Got next segment. Inline: false Dec 26 23:49:08.014165588 (512h20m7.885248643s) [upload=449566]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:49:08.014171370 (512h20m7.885254415s) [upload=449566]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:49:08.257201237 (512h20m8.128284282s) [seg_pos={0 0}, upload=449566]: Begin upload segment... Dec 26 23:49:08.257258957 (512h20m8.128341992s) [seg_pos={0 0}, upload=449566]: Done begin upload segment. Dec 26 23:49:08.257303993 (512h20m8.128387028s) [noise=true , address=145.79.227.11:4730 , piece_id=72V3KUQPKT7GALQMCAC6FEKY4PJHZSP2JXSYTUBCMUSKWJOXGGWQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.257312909 (512h20m8.128395954s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.257432187 (512h20m8.128515232s) [noise=true , address=145.79.227.47:4267 , piece_id=BWYAZVPL3NP5CNOID5F3AWOQNVQAL7YFI5SUSTCZONF2AB5TUESA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258117112 (512h20m8.129200147s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258189880 (512h20m8.129272915s) [noise=true , address=77.90.31.21:4880 , piece_id=UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258511592 (512h20m8.129594638s) [noise=true , address=145.79.227.37:4565 , piece_id=GBRZTO6LEJRMIX4U7TJEVGITM7F4ERZE32N2SMYCALQ3NPH4B7HA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258538524 (512h20m8.129621559s) [noise=true , address=145.79.227.43:4158 , piece_id=REQ3ZDWCDETW45JCM4AMJS4JGWAX5BUYSYUUIXMI6EQJYQM26RZQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258886797 (512h20m8.129969842s) [noise=true , address=145.79.227.3:4149 , piece_id=E2TPD3CCIVPD4PWZ2BOOLEJPWOJNFN4RSMICXMWDLQFFGDPINMRA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.259294292 (512h20m8.130377327s) [noise=true , address=45.91.116.9:4262 , piece_id=IA5NAD5A3U4X3BDE5G6HIZEMATUBT4SI47SS4MS36TZTHB5VABIQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.259590747 (512h20m8.130673792s) [noise=true , address=145.79.227.49:4338 , piece_id=VG2UGTUWRF6GS7CAOFYFHU7R7EBREPFJLJ6H6EXBYN2YSF4CZUNQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260037418 (512h20m8.131120463s) [noise=true , address=145.79.227.23:4545 , piece_id=U7ZUHXUW7YDJFTWNYG62KK7IRM6CZPV2D54JPHCCYKKCTTMXVTXQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260143260 (512h20m8.131226305s) [noise=true , address=145.79.227.15:4874 , piece_id=L6PSSXHWTREKWVNQF63HA6I6OPYGOVUTAUS23A5TMJB7ZYHMF3VA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260894469 (512h20m8.131977515s) [noise=true , address=145.79.227.51:4411 , piece_id=MSU73BJNZBCGY2U7SO6VZEPT4TIXJ6WTQ444FJON5X365RGWWTZQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.261391386 (512h20m8.132474421s) [noise=true , address=45.91.116.13:4154 , piece_id=JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.261829600 (512h20m8.132912645s) [noise=true , address=45.91.116.6:4974 , piece_id=4PB7PMAG5W3ENIO22ZMW2JKDBNPI6HWF6N3X7B353IX62OL4FW2A, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.262531116 (512h20m8.133614151s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:09.873318071 (512h20m9.744401106s) [upload=449566]: Next returned nil segment Dec 26 23:49:09.873340975 (512h20m9.74442403s) [upload=449566]: Waiting for error group managing segments... Dec 26 23:49:10.261144033 (512h20m10.132227078s) [noise=true , address=145.79.227.43:4158 , piece_id=REQ3ZDWCDETW45JCM4AMJS4JGWAX5BUYSYUUIXMI6EQJYQM26RZQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.266861901 (512h20m10.137944946s) [noise=true , address=145.79.227.15:4874 , piece_id=L6PSSXHWTREKWVNQF63HA6I6OPYGOVUTAUS23A5TMJB7ZYHMF3VA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.270728854 (512h20m10.141811889s) [noise=true , address=145.79.227.47:4267 , piece_id=BWYAZVPL3NP5CNOID5F3AWOQNVQAL7YFI5SUSTCZONF2AB5TUESA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.274324330 (512h20m10.145407385s) [noise=true , address=145.79.227.23:4545 , piece_id=U7ZUHXUW7YDJFTWNYG62KK7IRM6CZPV2D54JPHCCYKKCTTMXVTXQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.274834943 (512h20m10.145917988s) [noise=true , address=145.79.227.51:4411 , piece_id=MSU73BJNZBCGY2U7SO6VZEPT4TIXJ6WTQ444FJON5X365RGWWTZQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.279578455 (512h20m10.1506615s) [noise=true , address=145.79.227.11:4730 , piece_id=72V3KUQPKT7GALQMCAC6FEKY4PJHZSP2JXSYTUBCMUSKWJOXGGWQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.280108365 (512h20m10.1511914s) [noise=true , address=145.79.227.49:4338 , piece_id=VG2UGTUWRF6GS7CAOFYFHU7R7EBREPFJLJ6H6EXBYN2YSF4CZUNQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.294136716 (512h20m10.165219761s) [noise=true , address=145.79.227.3:4149 , piece_id=E2TPD3CCIVPD4PWZ2BOOLEJPWOJNFN4RSMICXMWDLQFFGDPINMRA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.296661514 (512h20m10.167744549s) [noise=true , address=145.79.227.37:4565 , piece_id=GBRZTO6LEJRMIX4U7TJEVGITM7F4ERZE32N2SMYCALQ3NPH4B7HA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.431780672 (512h20m10.302863707s) [noise=true , address=77.90.31.21:4880 , piece_id=UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.486589888 (512h20m10.357672923s) [noise=true , address=45.91.116.6:4974 , piece_id=4PB7PMAG5W3ENIO22ZMW2JKDBNPI6HWF6N3X7B353IX62OL4FW2A, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.507362222 (512h20m10.378445257s) [noise=true , address=45.91.116.9:4262 , piece_id=IA5NAD5A3U4X3BDE5G6HIZEMATUBT4SI47SS4MS36TZTHB5VABIQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.509180464 (512h20m10.380263509s) [noise=true , address=45.91.116.13:4154 , piece_id=JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.509192306 (512h20m10.380275341s) [seg_pos={0 0}, upload=449566]: Segment reached optimal threshold of 13 pieces. Dec 26 23:49:10.509457201 (512h20m10.380540236s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:49:10.509464395 (512h20m10.38054743s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509471909 (512h20m10.380554944s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Dec 26 23:49:10.509482239 (512h20m10.380565274s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509670036 (512h20m10.380753071s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection Dec 26 23:49:10.509679033 (512h20m10.380762068s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509701667 (512h20m10.380784702s) [seg_pos={0 0}, upload=449566]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:49:10.509768304 (512h20m10.380851339s) [upload=449566]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:49:10.509773584 (512h20m10.380856619s) [upload=449566]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:49:10.805818109 (512h20m10.676901154s) [upload=449566]: Done uploading segments... Dec 26 23:49:10.805836805 (512h20m10.67691985s) [upload=449566]: Upload finished. err: <nil> Duration: 7.958891119s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 244.044446ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:50:49 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 244.070998ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [............X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX..] duration: 6.149506614s trace: https://jaeger.dev.storj.io/trace/105cfb5b0968a8f3 FAIL (2025-12-26 23:50:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA) context deadline exceeded; piecestore: (Node ID: 1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, Piece ID: UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A) context deadline exceeded ======LOGS====== Dec 26 23:49:08.013669624 (512h20m7.884752659s) [upload=449566]: Starting upload Dec 26 23:49:08.013722815 (512h20m7.88480585s) [upload=449566]: Done starting upload Dec 26 23:49:08.013923747 (512h20m7.885006792s) [upload=449566]: Uploading segments... Dec 26 23:49:08.014118418 (512h20m7.885201463s) [upload=449566]: Got next segment. Inline: false Dec 26 23:49:08.014165588 (512h20m7.885248643s) [upload=449566]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:49:08.014171370 (512h20m7.885254415s) [upload=449566]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:49:08.257201237 (512h20m8.128284282s) [seg_pos={0 0}, upload=449566]: Begin upload segment... Dec 26 23:49:08.257258957 (512h20m8.128341992s) [seg_pos={0 0}, upload=449566]: Done begin upload segment. Dec 26 23:49:08.257303993 (512h20m8.128387028s) [noise=true , address=145.79.227.11:4730 , piece_id=72V3KUQPKT7GALQMCAC6FEKY4PJHZSP2JXSYTUBCMUSKWJOXGGWQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.257312909 (512h20m8.128395954s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.257432187 (512h20m8.128515232s) [noise=true , address=145.79.227.47:4267 , piece_id=BWYAZVPL3NP5CNOID5F3AWOQNVQAL7YFI5SUSTCZONF2AB5TUESA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258117112 (512h20m8.129200147s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258189880 (512h20m8.129272915s) [noise=true , address=77.90.31.21:4880 , piece_id=UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258511592 (512h20m8.129594638s) [noise=true , address=145.79.227.37:4565 , piece_id=GBRZTO6LEJRMIX4U7TJEVGITM7F4ERZE32N2SMYCALQ3NPH4B7HA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258538524 (512h20m8.129621559s) [noise=true , address=145.79.227.43:4158 , piece_id=REQ3ZDWCDETW45JCM4AMJS4JGWAX5BUYSYUUIXMI6EQJYQM26RZQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258886797 (512h20m8.129969842s) [noise=true , address=145.79.227.3:4149 , piece_id=E2TPD3CCIVPD4PWZ2BOOLEJPWOJNFN4RSMICXMWDLQFFGDPINMRA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.259294292 (512h20m8.130377327s) [noise=true , address=45.91.116.9:4262 , piece_id=IA5NAD5A3U4X3BDE5G6HIZEMATUBT4SI47SS4MS36TZTHB5VABIQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.259590747 (512h20m8.130673792s) [noise=true , address=145.79.227.49:4338 , piece_id=VG2UGTUWRF6GS7CAOFYFHU7R7EBREPFJLJ6H6EXBYN2YSF4CZUNQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260037418 (512h20m8.131120463s) [noise=true , address=145.79.227.23:4545 , piece_id=U7ZUHXUW7YDJFTWNYG62KK7IRM6CZPV2D54JPHCCYKKCTTMXVTXQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260143260 (512h20m8.131226305s) [noise=true , address=145.79.227.15:4874 , piece_id=L6PSSXHWTREKWVNQF63HA6I6OPYGOVUTAUS23A5TMJB7ZYHMF3VA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260894469 (512h20m8.131977515s) [noise=true , address=145.79.227.51:4411 , piece_id=MSU73BJNZBCGY2U7SO6VZEPT4TIXJ6WTQ444FJON5X365RGWWTZQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.261391386 (512h20m8.132474421s) [noise=true , address=45.91.116.13:4154 , piece_id=JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.261829600 (512h20m8.132912645s) [noise=true , address=45.91.116.6:4974 , piece_id=4PB7PMAG5W3ENIO22ZMW2JKDBNPI6HWF6N3X7B353IX62OL4FW2A, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.262531116 (512h20m8.133614151s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:09.873318071 (512h20m9.744401106s) [upload=449566]: Next returned nil segment Dec 26 23:49:09.873340975 (512h20m9.74442403s) [upload=449566]: Waiting for error group managing segments... Dec 26 23:49:10.261144033 (512h20m10.132227078s) [noise=true , address=145.79.227.43:4158 , piece_id=REQ3ZDWCDETW45JCM4AMJS4JGWAX5BUYSYUUIXMI6EQJYQM26RZQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.266861901 (512h20m10.137944946s) [noise=true , address=145.79.227.15:4874 , piece_id=L6PSSXHWTREKWVNQF63HA6I6OPYGOVUTAUS23A5TMJB7ZYHMF3VA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.270728854 (512h20m10.141811889s) [noise=true , address=145.79.227.47:4267 , piece_id=BWYAZVPL3NP5CNOID5F3AWOQNVQAL7YFI5SUSTCZONF2AB5TUESA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.274324330 (512h20m10.145407385s) [noise=true , address=145.79.227.23:4545 , piece_id=U7ZUHXUW7YDJFTWNYG62KK7IRM6CZPV2D54JPHCCYKKCTTMXVTXQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.274834943 (512h20m10.145917988s) [noise=true , address=145.79.227.51:4411 , piece_id=MSU73BJNZBCGY2U7SO6VZEPT4TIXJ6WTQ444FJON5X365RGWWTZQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.279578455 (512h20m10.1506615s) [noise=true , address=145.79.227.11:4730 , piece_id=72V3KUQPKT7GALQMCAC6FEKY4PJHZSP2JXSYTUBCMUSKWJOXGGWQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.280108365 (512h20m10.1511914s) [noise=true , address=145.79.227.49:4338 , piece_id=VG2UGTUWRF6GS7CAOFYFHU7R7EBREPFJLJ6H6EXBYN2YSF4CZUNQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.294136716 (512h20m10.165219761s) [noise=true , address=145.79.227.3:4149 , piece_id=E2TPD3CCIVPD4PWZ2BOOLEJPWOJNFN4RSMICXMWDLQFFGDPINMRA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.296661514 (512h20m10.167744549s) [noise=true , address=145.79.227.37:4565 , piece_id=GBRZTO6LEJRMIX4U7TJEVGITM7F4ERZE32N2SMYCALQ3NPH4B7HA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.431780672 (512h20m10.302863707s) [noise=true , address=77.90.31.21:4880 , piece_id=UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.486589888 (512h20m10.357672923s) [noise=true , address=45.91.116.6:4974 , piece_id=4PB7PMAG5W3ENIO22ZMW2JKDBNPI6HWF6N3X7B353IX62OL4FW2A, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.507362222 (512h20m10.378445257s) [noise=true , address=45.91.116.9:4262 , piece_id=IA5NAD5A3U4X3BDE5G6HIZEMATUBT4SI47SS4MS36TZTHB5VABIQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.509180464 (512h20m10.380263509s) [noise=true , address=45.91.116.13:4154 , piece_id=JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.509192306 (512h20m10.380275341s) [seg_pos={0 0}, upload=449566]: Segment reached optimal threshold of 13 pieces. Dec 26 23:49:10.509457201 (512h20m10.380540236s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:49:10.509464395 (512h20m10.38054743s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509471909 (512h20m10.380554944s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Dec 26 23:49:10.509482239 (512h20m10.380565274s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509670036 (512h20m10.380753071s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection Dec 26 23:49:10.509679033 (512h20m10.380762068s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509701667 (512h20m10.380784702s) [seg_pos={0 0}, upload=449566]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:49:10.509768304 (512h20m10.380851339s) [upload=449566]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:49:10.509773584 (512h20m10.380856619s) [upload=449566]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:49:10.805818109 (512h20m10.676901154s) [upload=449566]: Done uploading segments... Dec 26 23:49:10.805836805 (512h20m10.67691985s) [upload=449566]: Upload finished. err: <nil> Duration: 6.149532554s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 244.329218ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:50:05 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 244.380828ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.............X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX.] duration: 9.884109619s trace: https://jaeger.dev.storj.io/trace/105cfb5b0968a8f3 FAIL (2025-12-26 23:49:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA) context deadline exceeded; piecestore: (Node ID: 1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, Piece ID: UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A) context deadline exceeded ======LOGS====== Dec 26 23:49:08.013669624 (512h20m7.884752659s) [upload=449566]: Starting upload Dec 26 23:49:08.013722815 (512h20m7.88480585s) [upload=449566]: Done starting upload Dec 26 23:49:08.013923747 (512h20m7.885006792s) [upload=449566]: Uploading segments... Dec 26 23:49:08.014118418 (512h20m7.885201463s) [upload=449566]: Got next segment. Inline: false Dec 26 23:49:08.014165588 (512h20m7.885248643s) [upload=449566]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:49:08.014171370 (512h20m7.885254415s) [upload=449566]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:49:08.257201237 (512h20m8.128284282s) [seg_pos={0 0}, upload=449566]: Begin upload segment... Dec 26 23:49:08.257258957 (512h20m8.128341992s) [seg_pos={0 0}, upload=449566]: Done begin upload segment. Dec 26 23:49:08.257303993 (512h20m8.128387028s) [noise=true , address=145.79.227.11:4730 , piece_id=72V3KUQPKT7GALQMCAC6FEKY4PJHZSP2JXSYTUBCMUSKWJOXGGWQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.257312909 (512h20m8.128395954s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.257432187 (512h20m8.128515232s) [noise=true , address=145.79.227.47:4267 , piece_id=BWYAZVPL3NP5CNOID5F3AWOQNVQAL7YFI5SUSTCZONF2AB5TUESA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258117112 (512h20m8.129200147s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258189880 (512h20m8.129272915s) [noise=true , address=77.90.31.21:4880 , piece_id=UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258511592 (512h20m8.129594638s) [noise=true , address=145.79.227.37:4565 , piece_id=GBRZTO6LEJRMIX4U7TJEVGITM7F4ERZE32N2SMYCALQ3NPH4B7HA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258538524 (512h20m8.129621559s) [noise=true , address=145.79.227.43:4158 , piece_id=REQ3ZDWCDETW45JCM4AMJS4JGWAX5BUYSYUUIXMI6EQJYQM26RZQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258886797 (512h20m8.129969842s) [noise=true , address=145.79.227.3:4149 , piece_id=E2TPD3CCIVPD4PWZ2BOOLEJPWOJNFN4RSMICXMWDLQFFGDPINMRA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.259294292 (512h20m8.130377327s) [noise=true , address=45.91.116.9:4262 , piece_id=IA5NAD5A3U4X3BDE5G6HIZEMATUBT4SI47SS4MS36TZTHB5VABIQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.259590747 (512h20m8.130673792s) [noise=true , address=145.79.227.49:4338 , piece_id=VG2UGTUWRF6GS7CAOFYFHU7R7EBREPFJLJ6H6EXBYN2YSF4CZUNQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260037418 (512h20m8.131120463s) [noise=true , address=145.79.227.23:4545 , piece_id=U7ZUHXUW7YDJFTWNYG62KK7IRM6CZPV2D54JPHCCYKKCTTMXVTXQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260143260 (512h20m8.131226305s) [noise=true , address=145.79.227.15:4874 , piece_id=L6PSSXHWTREKWVNQF63HA6I6OPYGOVUTAUS23A5TMJB7ZYHMF3VA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260894469 (512h20m8.131977515s) [noise=true , address=145.79.227.51:4411 , piece_id=MSU73BJNZBCGY2U7SO6VZEPT4TIXJ6WTQ444FJON5X365RGWWTZQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.261391386 (512h20m8.132474421s) [noise=true , address=45.91.116.13:4154 , piece_id=JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.261829600 (512h20m8.132912645s) [noise=true , address=45.91.116.6:4974 , piece_id=4PB7PMAG5W3ENIO22ZMW2JKDBNPI6HWF6N3X7B353IX62OL4FW2A, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.262531116 (512h20m8.133614151s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:09.873318071 (512h20m9.744401106s) [upload=449566]: Next returned nil segment Dec 26 23:49:09.873340975 (512h20m9.74442403s) [upload=449566]: Waiting for error group managing segments... Dec 26 23:49:10.261144033 (512h20m10.132227078s) [noise=true , address=145.79.227.43:4158 , piece_id=REQ3ZDWCDETW45JCM4AMJS4JGWAX5BUYSYUUIXMI6EQJYQM26RZQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.266861901 (512h20m10.137944946s) [noise=true , address=145.79.227.15:4874 , piece_id=L6PSSXHWTREKWVNQF63HA6I6OPYGOVUTAUS23A5TMJB7ZYHMF3VA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.270728854 (512h20m10.141811889s) [noise=true , address=145.79.227.47:4267 , piece_id=BWYAZVPL3NP5CNOID5F3AWOQNVQAL7YFI5SUSTCZONF2AB5TUESA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.274324330 (512h20m10.145407385s) [noise=true , address=145.79.227.23:4545 , piece_id=U7ZUHXUW7YDJFTWNYG62KK7IRM6CZPV2D54JPHCCYKKCTTMXVTXQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.274834943 (512h20m10.145917988s) [noise=true , address=145.79.227.51:4411 , piece_id=MSU73BJNZBCGY2U7SO6VZEPT4TIXJ6WTQ444FJON5X365RGWWTZQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.279578455 (512h20m10.1506615s) [noise=true , address=145.79.227.11:4730 , piece_id=72V3KUQPKT7GALQMCAC6FEKY4PJHZSP2JXSYTUBCMUSKWJOXGGWQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.280108365 (512h20m10.1511914s) [noise=true , address=145.79.227.49:4338 , piece_id=VG2UGTUWRF6GS7CAOFYFHU7R7EBREPFJLJ6H6EXBYN2YSF4CZUNQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.294136716 (512h20m10.165219761s) [noise=true , address=145.79.227.3:4149 , piece_id=E2TPD3CCIVPD4PWZ2BOOLEJPWOJNFN4RSMICXMWDLQFFGDPINMRA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.296661514 (512h20m10.167744549s) [noise=true , address=145.79.227.37:4565 , piece_id=GBRZTO6LEJRMIX4U7TJEVGITM7F4ERZE32N2SMYCALQ3NPH4B7HA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.431780672 (512h20m10.302863707s) [noise=true , address=77.90.31.21:4880 , piece_id=UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.486589888 (512h20m10.357672923s) [noise=true , address=45.91.116.6:4974 , piece_id=4PB7PMAG5W3ENIO22ZMW2JKDBNPI6HWF6N3X7B353IX62OL4FW2A, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.507362222 (512h20m10.378445257s) [noise=true , address=45.91.116.9:4262 , piece_id=IA5NAD5A3U4X3BDE5G6HIZEMATUBT4SI47SS4MS36TZTHB5VABIQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.509180464 (512h20m10.380263509s) [noise=true , address=45.91.116.13:4154 , piece_id=JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.509192306 (512h20m10.380275341s) [seg_pos={0 0}, upload=449566]: Segment reached optimal threshold of 13 pieces. Dec 26 23:49:10.509457201 (512h20m10.380540236s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:49:10.509464395 (512h20m10.38054743s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509471909 (512h20m10.380554944s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Dec 26 23:49:10.509482239 (512h20m10.380565274s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509670036 (512h20m10.380753071s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection Dec 26 23:49:10.509679033 (512h20m10.380762068s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509701667 (512h20m10.380784702s) [seg_pos={0 0}, upload=449566]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:49:10.509768304 (512h20m10.380851339s) [upload=449566]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:49:10.509773584 (512h20m10.380856619s) [upload=449566]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:49:10.805818109 (512h20m10.676901154s) [upload=449566]: Done uploading segments... Dec 26 23:49:10.805836805 (512h20m10.67691985s) [upload=449566]: Upload finished. err: <nil> Duration: 9.884123596s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [..............X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.XX] duration: 30.0037045s trace: https://jaeger.dev.storj.io/trace/105cfb5b0968a8f3 FAIL (2025-12-26 23:49:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA) context deadline exceeded; piecestore: (Node ID: 1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, Piece ID: UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A) context deadline exceeded ======LOGS====== Dec 26 23:49:08.013669624 (512h20m7.884752659s) [upload=449566]: Starting upload Dec 26 23:49:08.013722815 (512h20m7.88480585s) [upload=449566]: Done starting upload Dec 26 23:49:08.013923747 (512h20m7.885006792s) [upload=449566]: Uploading segments... Dec 26 23:49:08.014118418 (512h20m7.885201463s) [upload=449566]: Got next segment. Inline: false Dec 26 23:49:08.014165588 (512h20m7.885248643s) [upload=449566]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:49:08.014171370 (512h20m7.885254415s) [upload=449566]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:49:08.257201237 (512h20m8.128284282s) [seg_pos={0 0}, upload=449566]: Begin upload segment... Dec 26 23:49:08.257258957 (512h20m8.128341992s) [seg_pos={0 0}, upload=449566]: Done begin upload segment. Dec 26 23:49:08.257303993 (512h20m8.128387028s) [noise=true , address=145.79.227.11:4730 , piece_id=72V3KUQPKT7GALQMCAC6FEKY4PJHZSP2JXSYTUBCMUSKWJOXGGWQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.257312909 (512h20m8.128395954s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.257432187 (512h20m8.128515232s) [noise=true , address=145.79.227.47:4267 , piece_id=BWYAZVPL3NP5CNOID5F3AWOQNVQAL7YFI5SUSTCZONF2AB5TUESA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258117112 (512h20m8.129200147s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258189880 (512h20m8.129272915s) [noise=true , address=77.90.31.21:4880 , piece_id=UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258511592 (512h20m8.129594638s) [noise=true , address=145.79.227.37:4565 , piece_id=GBRZTO6LEJRMIX4U7TJEVGITM7F4ERZE32N2SMYCALQ3NPH4B7HA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258538524 (512h20m8.129621559s) [noise=true , address=145.79.227.43:4158 , piece_id=REQ3ZDWCDETW45JCM4AMJS4JGWAX5BUYSYUUIXMI6EQJYQM26RZQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.258886797 (512h20m8.129969842s) [noise=true , address=145.79.227.3:4149 , piece_id=E2TPD3CCIVPD4PWZ2BOOLEJPWOJNFN4RSMICXMWDLQFFGDPINMRA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.259294292 (512h20m8.130377327s) [noise=true , address=45.91.116.9:4262 , piece_id=IA5NAD5A3U4X3BDE5G6HIZEMATUBT4SI47SS4MS36TZTHB5VABIQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.259590747 (512h20m8.130673792s) [noise=true , address=145.79.227.49:4338 , piece_id=VG2UGTUWRF6GS7CAOFYFHU7R7EBREPFJLJ6H6EXBYN2YSF4CZUNQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260037418 (512h20m8.131120463s) [noise=true , address=145.79.227.23:4545 , piece_id=U7ZUHXUW7YDJFTWNYG62KK7IRM6CZPV2D54JPHCCYKKCTTMXVTXQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260143260 (512h20m8.131226305s) [noise=true , address=145.79.227.15:4874 , piece_id=L6PSSXHWTREKWVNQF63HA6I6OPYGOVUTAUS23A5TMJB7ZYHMF3VA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.260894469 (512h20m8.131977515s) [noise=true , address=145.79.227.51:4411 , piece_id=MSU73BJNZBCGY2U7SO6VZEPT4TIXJ6WTQ444FJON5X365RGWWTZQ, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.261391386 (512h20m8.132474421s) [noise=true , address=45.91.116.13:4154 , piece_id=JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.261829600 (512h20m8.132912645s) [noise=true , address=45.91.116.6:4974 , piece_id=4PB7PMAG5W3ENIO22ZMW2JKDBNPI6HWF6N3X7B353IX62OL4FW2A, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:08.262531116 (512h20m8.133614151s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Uploading piece... Dec 26 23:49:09.873318071 (512h20m9.744401106s) [upload=449566]: Next returned nil segment Dec 26 23:49:09.873340975 (512h20m9.74442403s) [upload=449566]: Waiting for error group managing segments... Dec 26 23:49:10.261144033 (512h20m10.132227078s) [noise=true , address=145.79.227.43:4158 , piece_id=REQ3ZDWCDETW45JCM4AMJS4JGWAX5BUYSYUUIXMI6EQJYQM26RZQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.266861901 (512h20m10.137944946s) [noise=true , address=145.79.227.15:4874 , piece_id=L6PSSXHWTREKWVNQF63HA6I6OPYGOVUTAUS23A5TMJB7ZYHMF3VA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.270728854 (512h20m10.141811889s) [noise=true , address=145.79.227.47:4267 , piece_id=BWYAZVPL3NP5CNOID5F3AWOQNVQAL7YFI5SUSTCZONF2AB5TUESA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.274324330 (512h20m10.145407385s) [noise=true , address=145.79.227.23:4545 , piece_id=U7ZUHXUW7YDJFTWNYG62KK7IRM6CZPV2D54JPHCCYKKCTTMXVTXQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.274834943 (512h20m10.145917988s) [noise=true , address=145.79.227.51:4411 , piece_id=MSU73BJNZBCGY2U7SO6VZEPT4TIXJ6WTQ444FJON5X365RGWWTZQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.279578455 (512h20m10.1506615s) [noise=true , address=145.79.227.11:4730 , piece_id=72V3KUQPKT7GALQMCAC6FEKY4PJHZSP2JXSYTUBCMUSKWJOXGGWQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.280108365 (512h20m10.1511914s) [noise=true , address=145.79.227.49:4338 , piece_id=VG2UGTUWRF6GS7CAOFYFHU7R7EBREPFJLJ6H6EXBYN2YSF4CZUNQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.294136716 (512h20m10.165219761s) [noise=true , address=145.79.227.3:4149 , piece_id=E2TPD3CCIVPD4PWZ2BOOLEJPWOJNFN4RSMICXMWDLQFFGDPINMRA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.296661514 (512h20m10.167744549s) [noise=true , address=145.79.227.37:4565 , piece_id=GBRZTO6LEJRMIX4U7TJEVGITM7F4ERZE32N2SMYCALQ3NPH4B7HA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.431780672 (512h20m10.302863707s) [noise=true , address=77.90.31.21:4880 , piece_id=UM4HLA6GQIAPYCPPOHAMVH43VTXXNW7TRLQ6PLYGJD47UO5R7M5A, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.486589888 (512h20m10.357672923s) [noise=true , address=45.91.116.6:4974 , piece_id=4PB7PMAG5W3ENIO22ZMW2JKDBNPI6HWF6N3X7B353IX62OL4FW2A, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.507362222 (512h20m10.378445257s) [noise=true , address=45.91.116.9:4262 , piece_id=IA5NAD5A3U4X3BDE5G6HIZEMATUBT4SI47SS4MS36TZTHB5VABIQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.509180464 (512h20m10.380263509s) [noise=true , address=45.91.116.13:4154 , piece_id=JBZ6HCL54IAMTI5EPP3MCE5FO24PJOO3OGWBGF7SNYNWC55AKCAA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: <nil> Dec 26 23:49:10.509192306 (512h20m10.380275341s) [seg_pos={0 0}, upload=449566]: Segment reached optimal threshold of 13 pieces. Dec 26 23:49:10.509457201 (512h20m10.380540236s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Dec 26 23:49:10.509464395 (512h20m10.38054743s) [noise=true , address=45.91.116.8:4047 , piece_id=J2RQC7VRAVHIU4MYYJ5R3YPKYRPVV6MEKWLY5XQPKAETZYFEP5BQ, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509471909 (512h20m10.380554944s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Dec 26 23:49:10.509482239 (512h20m10.380565274s) [noise=true , address=77.90.31.10:4520 , piece_id=PRIVQ5EFSJBGS7HC3ZSS4UWQOBQFXF2JH5MXZJFIWOS5E4PHN6EA, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509670036 (512h20m10.380753071s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41602->45.91.116.4:4937: use of closed network connection Dec 26 23:49:10.509679033 (512h20m10.380762068s) [noise=true , address=45.91.116.4:4937 , piece_id=DDKZG5QGBU3YC7RKNZ5JG3OMDM55CSGEBM2GKEIOIWRNTLV2ZOEA, seg_pos={0 0}, upload=449566]: Long tail cancellation due to optimal threshold Dec 26 23:49:10.509701667 (512h20m10.380784702s) [seg_pos={0 0}, upload=449566]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:49:10.509768304 (512h20m10.380851339s) [upload=449566]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:49:10.509773584 (512h20m10.380856619s) [upload=449566]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:49:10.805818109 (512h20m10.676901154s) [upload=449566]: Done uploading segments... Dec 26 23:49:10.805836805 (512h20m10.67691985s) [upload=449566]: Upload finished. err: <nil> Duration: 30.003729909s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 250.24687ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:49:34 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 250.272739ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 256.60143ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:49:14 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 256.630736ms edge linksharing bgp eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.27806253s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:48:46 UTC) orbiter warning: context deadline exceeded Duration: 6.278086606s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [...............X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.X] duration: 30.002841145s trace: https://jaeger.dev.storj.io/trace/434e35ca05241a97 FAIL (2025-12-26 23:48:36 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 26 23:48:06.734135446 (512h19m6.605218481s) [upload=449547]: Starting upload Dec 26 23:48:06.734250465 (512h19m6.6053335s) [upload=449547]: Done starting upload Dec 26 23:48:06.734397054 (512h19m6.605480089s) [upload=449547]: Uploading segments... Dec 26 23:48:06.734491303 (512h19m6.605574338s) [upload=449547]: Got next segment. Inline: false Dec 26 23:48:06.734544275 (512h19m6.60562732s) [upload=449547]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:48:06.734550296 (512h19m6.605633331s) [upload=449547]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:48:06.979559844 (512h19m6.850642899s) [seg_pos={0 0}, upload=449547]: Begin upload segment... Dec 26 23:48:06.979607746 (512h19m6.850690781s) [seg_pos={0 0}, upload=449547]: Done begin upload segment. Dec 26 23:48:06.979639537 (512h19m6.850722572s) [noise=true , address=45.91.116.5:4298 , piece_id=EMECQHDD4GTXD2VZIUDN75GTMXKX4FRRXCSNY5NZLR2SVIR47Q5Q, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.979709670 (512h19m6.850792715s) [noise=true , address=77.90.31.15:4700 , piece_id=6GW2MQSDVBCMQJUGQLL6FS7NGUTJVFEE3VNIBYEISBBEW6ME5CRA, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.980137384 (512h19m6.851220429s) [noise=true , address=145.79.227.46:4602 , piece_id=BEEMNHQCIZQHBY4DOTJ64IYPJUNZJKBUD23IVR5ZYH6WZLIUJAHQ, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.980461962 (512h19m6.851544997s) [noise=true , address=145.79.227.42:4122 , piece_id=IG6P54LQSVNQXUQX5JRJ3T5BLRESUV57RQN5KNJRT2AKQLPQSRNQ, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.981501431 (512h19m6.852584467s) [noise=true , address=45.91.116.4:4940 , piece_id=6EWQQL6XEVZALH57AECYA2XHAHWUHUC7NXK3VKCJMWNWOHHU7TWA, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.981902636 (512h19m6.852985671s) [noise=true , address=145.79.227.45:4231 , piece_id=P6YCVYB6RZB2IDCSQVS2TM4RG2XJ7APS7FFJF45TVXXIJGQ7FPTA, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.982940261 (512h19m6.854023296s) [noise=true , address=45.91.116.7:4010 , piece_id=27Y6MXCTV63DNTIDD7Z5LMZV32YGTCPGVUY6XEHINF4J262BV5EQ, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.983722150 (512h19m6.854805195s) [noise=true , address=145.79.227.11:4729 , piece_id=Z2CFY6B6B27KAAHMRQUTCEF75STABEZRNEUUMMTAGAT6Q75JEPKQ, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.984799452 (512h19m6.855882487s) [noise=true , address=77.90.31.17:4770 , piece_id=AVQLTMIVRDZP4UBJQP5UJL2Y4GRTSUEOGKXYIG4FKFH3VWO44THA, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.984908179 (512h19m6.855991214s) [noise=true , address=45.91.116.10:4084 , piece_id=7YHRW3DSMQ25AFFFZG4H3SVJSERBINESFZA3MLAGR5E6KAECPR6A, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.985687372 (512h19m6.856770417s) [noise=true , address=45.91.116.9:4264 , piece_id=TLEUKHHHS3UUMXUWFU74JP27NFSBS6RE3JZVV2TAKB4VQJEGCLZQ, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.986072385 (512h19m6.85715545s) [noise=true , address=145.79.227.7:4295 , piece_id=K7EGCEBBYO5T3NXJUPL7O3OSGBFR5KR2QG34QA2KCIQ3ZRRFIH6Q, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.987188099 (512h19m6.858271145s) [noise=true , address=145.79.227.6:4257 , piece_id=5C6LCTOE5OLO2U3QGM7H7P43INREABOTBFRW3VSZEKTVBYPZZSLA, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.989211634 (512h19m6.860294669s) [noise=true , address=145.79.227.38:4977 , piece_id=JZIFS2PTIDTTNQTS5RO342GNBXJGDGMTLMM7VWZJENARDSKNY6OA, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.991475564 (512h19m6.862558609s) [noise=true , address=145.79.227.48:4303 , piece_id=DY4XFSKAPICGPOTEOA6X64UDNTPPTUICL674EIY23FFDN3FNBPXQ, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:06.995867788 (512h19m6.866950833s) [noise=true , address=145.79.227.15:4875 , piece_id=JBJYCNZH4VOE2ISIMEQH6R4VUOPCMURVU55QM63KM5N77JWHDTXA, seg_pos={0 0}, upload=449547]: Uploading piece... Dec 26 23:48:08.695422571 (512h19m8.566505606s) [upload=449547]: Next returned nil segment Dec 26 23:48:08.695440746 (512h19m8.566523801s) [upload=449547]: Waiting for error group managing segments... Dec 26 23:48:08.973675830 (512h19m8.844758865s) [noise=true , address=145.79.227.7:4295 , piece_id=K7EGCEBBYO5T3NXJUPL7O3OSGBFR5KR2QG34QA2KCIQ3ZRRFIH6Q, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:08.978717109 (512h19m8.849800144s) [noise=true , address=145.79.227.38:4977 , piece_id=JZIFS2PTIDTTNQTS5RO342GNBXJGDGMTLMM7VWZJENARDSKNY6OA, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:08.990672935 (512h19m8.86175597s) [noise=true , address=145.79.227.15:4875 , piece_id=JBJYCNZH4VOE2ISIMEQH6R4VUOPCMURVU55QM63KM5N77JWHDTXA, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.001181746 (512h19m8.872264811s) [noise=true , address=145.79.227.46:4602 , piece_id=BEEMNHQCIZQHBY4DOTJ64IYPJUNZJKBUD23IVR5ZYH6WZLIUJAHQ, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.004082509 (512h19m8.875165544s) [noise=true , address=145.79.227.45:4231 , piece_id=P6YCVYB6RZB2IDCSQVS2TM4RG2XJ7APS7FFJF45TVXXIJGQ7FPTA, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.008139474 (512h19m8.879222509s) [noise=true , address=145.79.227.42:4122 , piece_id=IG6P54LQSVNQXUQX5JRJ3T5BLRESUV57RQN5KNJRT2AKQLPQSRNQ, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.008190261 (512h19m8.879273306s) [noise=true , address=145.79.227.6:4257 , piece_id=5C6LCTOE5OLO2U3QGM7H7P43INREABOTBFRW3VSZEKTVBYPZZSLA, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.008703508 (512h19m8.879786553s) [noise=true , address=145.79.227.11:4729 , piece_id=Z2CFY6B6B27KAAHMRQUTCEF75STABEZRNEUUMMTAGAT6Q75JEPKQ, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.010669131 (512h19m8.881752176s) [noise=true , address=145.79.227.48:4303 , piece_id=DY4XFSKAPICGPOTEOA6X64UDNTPPTUICL674EIY23FFDN3FNBPXQ, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.183071282 (512h19m9.054154327s) [noise=true , address=77.90.31.17:4770 , piece_id=AVQLTMIVRDZP4UBJQP5UJL2Y4GRTSUEOGKXYIG4FKFH3VWO44THA, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.209636126 (512h19m9.080719171s) [noise=true , address=45.91.116.7:4010 , piece_id=27Y6MXCTV63DNTIDD7Z5LMZV32YGTCPGVUY6XEHINF4J262BV5EQ, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.227958906 (512h19m9.099041991s) [noise=true , address=45.91.116.4:4940 , piece_id=6EWQQL6XEVZALH57AECYA2XHAHWUHUC7NXK3VKCJMWNWOHHU7TWA, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.230239658 (512h19m9.101322693s) [noise=true , address=45.91.116.5:4298 , piece_id=EMECQHDD4GTXD2VZIUDN75GTMXKX4FRRXCSNY5NZLR2SVIR47Q5Q, seg_pos={0 0}, upload=449547]: Done uploading piece. err: <nil> Dec 26 23:48:09.230251541 (512h19m9.101334576s) [seg_pos={0 0}, upload=449547]: Segment reached optimal threshold of 13 pieces. Dec 26 23:48:09.230549979 (512h19m9.101633014s) [noise=true , address=45.91.116.9:4264 , piece_id=TLEUKHHHS3UUMXUWFU74JP27NFSBS6RE3JZVV2TAKB4VQJEGCLZQ, seg_pos={0 0}, upload=449547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m): protocol: expected piece hash; context canceled; EOF Dec 26 23:48:09.230557674 (512h19m9.101640709s) [noise=true , address=45.91.116.9:4264 , piece_id=TLEUKHHHS3UUMXUWFU74JP27NFSBS6RE3JZVV2TAKB4VQJEGCLZQ, seg_pos={0 0}, upload=449547]: Long tail cancellation due to optimal threshold Dec 26 23:48:09.230748247 (512h19m9.101831292s) [noise=true , address=77.90.31.15:4700 , piece_id=6GW2MQSDVBCMQJUGQLL6FS7NGUTJVFEE3VNIBYEISBBEW6ME5CRA, seg_pos={0 0}, upload=449547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124p2J9pP2rYD3cM73PQNEp6Ykgmpq2xi6aK8wnC6JTiLvHRb7q): protocol: expected piece hash; context canceled; EOF Dec 26 23:48:09.230777583 (512h19m9.101860618s) [noise=true , address=77.90.31.15:4700 , piece_id=6GW2MQSDVBCMQJUGQLL6FS7NGUTJVFEE3VNIBYEISBBEW6ME5CRA, seg_pos={0 0}, upload=449547]: Long tail cancellation due to optimal threshold Dec 26 23:48:09.230795437 (512h19m9.101878482s) [noise=true , address=45.91.116.10:4084 , piece_id=7YHRW3DSMQ25AFFFZG4H3SVJSERBINESFZA3MLAGR5E6KAECPR6A, seg_pos={0 0}, upload=449547]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj): protocol: expected piece hash; context canceled; EOF Dec 26 23:48:09.230804173 (512h19m9.101887208s) [noise=true , address=45.91.116.10:4084 , piece_id=7YHRW3DSMQ25AFFFZG4H3SVJSERBINESFZA3MLAGR5E6KAECPR6A, seg_pos={0 0}, upload=449547]: Long tail cancellation due to optimal threshold Dec 26 23:48:09.230823340 (512h19m9.101906375s) [seg_pos={0 0}, upload=449547]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:48:09.230900637 (512h19m9.101983672s) [upload=449547]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:48:09.230904895 (512h19m9.10198793s) [upload=449547]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:48:09.527871156 (512h19m9.398954191s) [upload=449547]: Done uploading segments... Dec 26 23:48:09.527900070 (512h19m9.398983115s) [upload=449547]: Upload finished. err: <nil> Duration: 30.00286428s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 253.788934ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:48:22 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 253.81303ms edge linksharing bgp eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.006428687s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:47:53 UTC) orbiter warning: context deadline exceeded Duration: 6.006452221s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [................X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X.] duration: 6.040049096s trace: https://jaeger.dev.storj.io/trace/653b8deffcd2a80c FAIL (2025-12-26 23:47:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 23:46:38.215646750 (512h17m38.086729785s) [upload=449528]: Starting upload Dec 26 23:46:38.215702836 (512h17m38.086785871s) [upload=449528]: Done starting upload Dec 26 23:46:38.215828195 (512h17m38.08691123s) [upload=449528]: Uploading segments... Dec 26 23:46:38.215961249 (512h17m38.087044284s) [upload=449528]: Got next segment. Inline: false Dec 26 23:46:38.215995865 (512h17m38.0870789s) [upload=449528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:46:38.216000253 (512h17m38.087083298s) [upload=449528]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:46:38.463715172 (512h17m38.334798207s) [seg_pos={0 0}, upload=449528]: Begin upload segment... Dec 26 23:46:38.463768322 (512h17m38.334851357s) [seg_pos={0 0}, upload=449528]: Done begin upload segment. Dec 26 23:46:38.463796186 (512h17m38.334879221s) [noise=true , address=145.79.227.45:4229 , piece_id=BIW2VLL22FXAVBLZBQ55227EX3CUXCL3N5ISXLDJOV2EBCOYNTXQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.463959056 (512h17m38.335042111s) [noise=true , address=45.91.116.8:4047 , piece_id=GHPTLRLI424J5P6VFHLEU6MJFQ4MZEAJQVKOXT52FKR4DXADN2WA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.464561543 (512h17m38.335644588s) [noise=true , address=45.91.116.9:4262 , piece_id=2U447THMDCMFNOAUSQJLUZPSSECTUWP77PCPCI5QJEUL3B5ISRWA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.464962556 (512h17m38.336045601s) [noise=true , address=45.91.116.11:4118 , piece_id=ZM4XTNMRDL54XKWIGGMW3PX235TSRCX4USMOD5TYLZ7HXR6LOKOA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.465408195 (512h17m38.33649124s) [noise=true , address=145.79.227.15:4874 , piece_id=3X652D5FTVKCI6RJAWR3VATAEI4OVHLBHVBBGA4Y7JVJ27SNRQAQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.465753503 (512h17m38.336836538s) [noise=true , address=145.79.227.13:4804 , piece_id=IFGJNURJ3EC3QKTRPYPPSCSX76IFDGIBCTIG6BLKJTXSW3YTGOZQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.466171529 (512h17m38.337254574s) [noise=true , address=77.90.31.18:4805 , piece_id=HJA6JOE3D6O3WUCU4LUJAAJVPBP7LBHE5OUJLRSAD3UKD2IM527A, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.466555580 (512h17m38.337638615s) [noise=true , address=77.90.31.11:4553 , piece_id=VLN6SP5HQ6K6ZH55TTQ3CHQ3K3PK244TATF2IQBSGJUV4GBK4O3Q, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.466935283 (512h17m38.338018328s) [noise=true , address=45.91.116.6:4975 , piece_id=QCSBAVHPFGU2R7NUVOORR76UY3CQYR2TLTMKP72AKEI3QZ7P5OPA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.467297312 (512h17m38.338380367s) [noise=true , address=145.79.227.46:4601 , piece_id=SXOVR7OFJRMOZKSMHUYP3MZE7SLT3RDL2FK2UAVXXRTQHC6NKA4A, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.468443875 (512h17m38.33952691s) [noise=true , address=145.79.227.33:4869 , piece_id=XRFBVRNRX2AYATSZQYN332EOCIGYODUJO3UPHRRVEUREFXIALNEQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.468549787 (512h17m38.339632822s) [noise=true , address=45.91.116.13:4156 , piece_id=K3NQVWXVO5GFWP6RGEZTRSFEGRDSKGDR52XZBD2XAXQGXO3MO7VA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.470384621 (512h17m38.341467666s) [noise=true , address=77.90.31.14:4664 , piece_id=6SIDM7QGY526I3LPX5P4XYVNTUAOD2ZM5KQHEWL4VUXUMF2P2RIA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.472804950 (512h17m38.343887985s) [noise=true , address=45.91.116.12:4228 , piece_id=GZNWANFP5TGEWUIPKBVBVUPHMCMXEJI7TE6AZUUSSAOM6PSDNMQQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.474203243 (512h17m38.345286278s) [noise=true , address=145.79.227.38:4978 , piece_id=PFIMF43TKO63AS7Z4RWLY32CROQ7CQW3FMETF7I756UXMNTFV6DQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.475474823 (512h17m38.346557868s) [noise=true , address=45.91.116.10:4083 , piece_id=L75J42W77B44T3TPVQHFQ3ED7W6QJOZ3TFHYVRQVQKHMFUALBZZQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:40.084204710 (512h17m39.955287755s) [upload=449528]: Next returned nil segment Dec 26 23:46:40.084227162 (512h17m39.955310197s) [upload=449528]: Waiting for error group managing segments... Dec 26 23:46:40.470501922 (512h17m40.341584957s) [noise=true , address=145.79.227.15:4874 , piece_id=3X652D5FTVKCI6RJAWR3VATAEI4OVHLBHVBBGA4Y7JVJ27SNRQAQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.470875011 (512h17m40.341958057s) [noise=true , address=145.79.227.13:4804 , piece_id=IFGJNURJ3EC3QKTRPYPPSCSX76IFDGIBCTIG6BLKJTXSW3YTGOZQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.472833451 (512h17m40.343916496s) [noise=true , address=145.79.227.45:4229 , piece_id=BIW2VLL22FXAVBLZBQ55227EX3CUXCL3N5ISXLDJOV2EBCOYNTXQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.479678174 (512h17m40.350761219s) [noise=true , address=145.79.227.46:4601 , piece_id=SXOVR7OFJRMOZKSMHUYP3MZE7SLT3RDL2FK2UAVXXRTQHC6NKA4A, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.481112976 (512h17m40.352196021s) [noise=true , address=145.79.227.38:4978 , piece_id=PFIMF43TKO63AS7Z4RWLY32CROQ7CQW3FMETF7I756UXMNTFV6DQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.684725327 (512h17m40.555808362s) [noise=true , address=145.79.227.33:4869 , piece_id=XRFBVRNRX2AYATSZQYN332EOCIGYODUJO3UPHRRVEUREFXIALNEQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.686443809 (512h17m40.557526844s) [noise=true , address=77.90.31.14:4664 , piece_id=6SIDM7QGY526I3LPX5P4XYVNTUAOD2ZM5KQHEWL4VUXUMF2P2RIA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.690326232 (512h17m40.561409267s) [noise=true , address=45.91.116.8:4047 , piece_id=GHPTLRLI424J5P6VFHLEU6MJFQ4MZEAJQVKOXT52FKR4DXADN2WA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.698471652 (512h17m40.569554687s) [noise=true , address=77.90.31.18:4805 , piece_id=HJA6JOE3D6O3WUCU4LUJAAJVPBP7LBHE5OUJLRSAD3UKD2IM527A, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.720836368 (512h17m40.591919413s) [noise=true , address=45.91.116.9:4262 , piece_id=2U447THMDCMFNOAUSQJLUZPSSECTUWP77PCPCI5QJEUL3B5ISRWA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.738749488 (512h17m40.609832523s) [noise=true , address=45.91.116.10:4083 , piece_id=L75J42W77B44T3TPVQHFQ3ED7W6QJOZ3TFHYVRQVQKHMFUALBZZQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.742740798 (512h17m40.613823843s) [noise=true , address=77.90.31.11:4553 , piece_id=VLN6SP5HQ6K6ZH55TTQ3CHQ3K3PK244TATF2IQBSGJUV4GBK4O3Q, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.793210560 (512h17m40.664293595s) [noise=true , address=45.91.116.13:4156 , piece_id=K3NQVWXVO5GFWP6RGEZTRSFEGRDSKGDR52XZBD2XAXQGXO3MO7VA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.793254373 (512h17m40.664337409s) [seg_pos={0 0}, upload=449528]: Segment reached optimal threshold of 13 pieces. Dec 26 23:46:40.793416341 (512h17m40.664499376s) [noise=true , address=45.91.116.12:4228 , piece_id=GZNWANFP5TGEWUIPKBVBVUPHMCMXEJI7TE6AZUUSSAOM6PSDNMQQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF Dec 26 23:46:40.793422804 (512h17m40.664505839s) [noise=true , address=45.91.116.12:4228 , piece_id=GZNWANFP5TGEWUIPKBVBVUPHMCMXEJI7TE6AZUUSSAOM6PSDNMQQ, seg_pos={0 0}, upload=449528]: Long tail cancellation due to optimal threshold Dec 26 23:46:40.793665436 (512h17m40.664748471s) [noise=true , address=45.91.116.6:4975 , piece_id=QCSBAVHPFGU2R7NUVOORR76UY3CQYR2TLTMKP72AKEI3QZ7P5OPA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Dec 26 23:46:40.793673451 (512h17m40.664756497s) [noise=true , address=45.91.116.6:4975 , piece_id=QCSBAVHPFGU2R7NUVOORR76UY3CQYR2TLTMKP72AKEI3QZ7P5OPA, seg_pos={0 0}, upload=449528]: Long tail cancellation due to optimal threshold Dec 26 23:46:40.793846872 (512h17m40.664929927s) [noise=true , address=45.91.116.11:4118 , piece_id=ZM4XTNMRDL54XKWIGGMW3PX235TSRCX4USMOD5TYLZ7HXR6LOKOA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:54658->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:54658->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54658->45.91.116.11:4118: use of closed network connection Dec 26 23:46:40.793854276 (512h17m40.664937311s) [noise=true , address=45.91.116.11:4118 , piece_id=ZM4XTNMRDL54XKWIGGMW3PX235TSRCX4USMOD5TYLZ7HXR6LOKOA, seg_pos={0 0}, upload=449528]: Long tail cancellation due to optimal threshold Dec 26 23:46:40.793869163 (512h17m40.664952199s) [seg_pos={0 0}, upload=449528]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:46:40.793929839 (512h17m40.665012884s) [upload=449528]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:46:40.793933646 (512h17m40.665016681s) [upload=449528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:46:41.092895268 (512h17m40.963978343s) [upload=449528]: Done uploading segments... Dec 26 23:46:41.092923722 (512h17m40.964006767s) [upload=449528]: Upload finished. err: <nil> Duration: 6.040075085s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 254.900661ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:47:14 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 254.931421ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.................X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..X] duration: 30.002632366s trace: https://jaeger.dev.storj.io/trace/653b8deffcd2a80c FAIL (2025-12-26 23:47:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 23:46:38.215646750 (512h17m38.086729785s) [upload=449528]: Starting upload Dec 26 23:46:38.215702836 (512h17m38.086785871s) [upload=449528]: Done starting upload Dec 26 23:46:38.215828195 (512h17m38.08691123s) [upload=449528]: Uploading segments... Dec 26 23:46:38.215961249 (512h17m38.087044284s) [upload=449528]: Got next segment. Inline: false Dec 26 23:46:38.215995865 (512h17m38.0870789s) [upload=449528]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:46:38.216000253 (512h17m38.087083298s) [upload=449528]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:46:38.463715172 (512h17m38.334798207s) [seg_pos={0 0}, upload=449528]: Begin upload segment... Dec 26 23:46:38.463768322 (512h17m38.334851357s) [seg_pos={0 0}, upload=449528]: Done begin upload segment. Dec 26 23:46:38.463796186 (512h17m38.334879221s) [noise=true , address=145.79.227.45:4229 , piece_id=BIW2VLL22FXAVBLZBQ55227EX3CUXCL3N5ISXLDJOV2EBCOYNTXQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.463959056 (512h17m38.335042111s) [noise=true , address=45.91.116.8:4047 , piece_id=GHPTLRLI424J5P6VFHLEU6MJFQ4MZEAJQVKOXT52FKR4DXADN2WA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.464561543 (512h17m38.335644588s) [noise=true , address=45.91.116.9:4262 , piece_id=2U447THMDCMFNOAUSQJLUZPSSECTUWP77PCPCI5QJEUL3B5ISRWA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.464962556 (512h17m38.336045601s) [noise=true , address=45.91.116.11:4118 , piece_id=ZM4XTNMRDL54XKWIGGMW3PX235TSRCX4USMOD5TYLZ7HXR6LOKOA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.465408195 (512h17m38.33649124s) [noise=true , address=145.79.227.15:4874 , piece_id=3X652D5FTVKCI6RJAWR3VATAEI4OVHLBHVBBGA4Y7JVJ27SNRQAQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.465753503 (512h17m38.336836538s) [noise=true , address=145.79.227.13:4804 , piece_id=IFGJNURJ3EC3QKTRPYPPSCSX76IFDGIBCTIG6BLKJTXSW3YTGOZQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.466171529 (512h17m38.337254574s) [noise=true , address=77.90.31.18:4805 , piece_id=HJA6JOE3D6O3WUCU4LUJAAJVPBP7LBHE5OUJLRSAD3UKD2IM527A, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.466555580 (512h17m38.337638615s) [noise=true , address=77.90.31.11:4553 , piece_id=VLN6SP5HQ6K6ZH55TTQ3CHQ3K3PK244TATF2IQBSGJUV4GBK4O3Q, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.466935283 (512h17m38.338018328s) [noise=true , address=45.91.116.6:4975 , piece_id=QCSBAVHPFGU2R7NUVOORR76UY3CQYR2TLTMKP72AKEI3QZ7P5OPA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.467297312 (512h17m38.338380367s) [noise=true , address=145.79.227.46:4601 , piece_id=SXOVR7OFJRMOZKSMHUYP3MZE7SLT3RDL2FK2UAVXXRTQHC6NKA4A, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.468443875 (512h17m38.33952691s) [noise=true , address=145.79.227.33:4869 , piece_id=XRFBVRNRX2AYATSZQYN332EOCIGYODUJO3UPHRRVEUREFXIALNEQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.468549787 (512h17m38.339632822s) [noise=true , address=45.91.116.13:4156 , piece_id=K3NQVWXVO5GFWP6RGEZTRSFEGRDSKGDR52XZBD2XAXQGXO3MO7VA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.470384621 (512h17m38.341467666s) [noise=true , address=77.90.31.14:4664 , piece_id=6SIDM7QGY526I3LPX5P4XYVNTUAOD2ZM5KQHEWL4VUXUMF2P2RIA, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.472804950 (512h17m38.343887985s) [noise=true , address=45.91.116.12:4228 , piece_id=GZNWANFP5TGEWUIPKBVBVUPHMCMXEJI7TE6AZUUSSAOM6PSDNMQQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.474203243 (512h17m38.345286278s) [noise=true , address=145.79.227.38:4978 , piece_id=PFIMF43TKO63AS7Z4RWLY32CROQ7CQW3FMETF7I756UXMNTFV6DQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:38.475474823 (512h17m38.346557868s) [noise=true , address=45.91.116.10:4083 , piece_id=L75J42W77B44T3TPVQHFQ3ED7W6QJOZ3TFHYVRQVQKHMFUALBZZQ, seg_pos={0 0}, upload=449528]: Uploading piece... Dec 26 23:46:40.084204710 (512h17m39.955287755s) [upload=449528]: Next returned nil segment Dec 26 23:46:40.084227162 (512h17m39.955310197s) [upload=449528]: Waiting for error group managing segments... Dec 26 23:46:40.470501922 (512h17m40.341584957s) [noise=true , address=145.79.227.15:4874 , piece_id=3X652D5FTVKCI6RJAWR3VATAEI4OVHLBHVBBGA4Y7JVJ27SNRQAQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.470875011 (512h17m40.341958057s) [noise=true , address=145.79.227.13:4804 , piece_id=IFGJNURJ3EC3QKTRPYPPSCSX76IFDGIBCTIG6BLKJTXSW3YTGOZQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.472833451 (512h17m40.343916496s) [noise=true , address=145.79.227.45:4229 , piece_id=BIW2VLL22FXAVBLZBQ55227EX3CUXCL3N5ISXLDJOV2EBCOYNTXQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.479678174 (512h17m40.350761219s) [noise=true , address=145.79.227.46:4601 , piece_id=SXOVR7OFJRMOZKSMHUYP3MZE7SLT3RDL2FK2UAVXXRTQHC6NKA4A, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.481112976 (512h17m40.352196021s) [noise=true , address=145.79.227.38:4978 , piece_id=PFIMF43TKO63AS7Z4RWLY32CROQ7CQW3FMETF7I756UXMNTFV6DQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.684725327 (512h17m40.555808362s) [noise=true , address=145.79.227.33:4869 , piece_id=XRFBVRNRX2AYATSZQYN332EOCIGYODUJO3UPHRRVEUREFXIALNEQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.686443809 (512h17m40.557526844s) [noise=true , address=77.90.31.14:4664 , piece_id=6SIDM7QGY526I3LPX5P4XYVNTUAOD2ZM5KQHEWL4VUXUMF2P2RIA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.690326232 (512h17m40.561409267s) [noise=true , address=45.91.116.8:4047 , piece_id=GHPTLRLI424J5P6VFHLEU6MJFQ4MZEAJQVKOXT52FKR4DXADN2WA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.698471652 (512h17m40.569554687s) [noise=true , address=77.90.31.18:4805 , piece_id=HJA6JOE3D6O3WUCU4LUJAAJVPBP7LBHE5OUJLRSAD3UKD2IM527A, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.720836368 (512h17m40.591919413s) [noise=true , address=45.91.116.9:4262 , piece_id=2U447THMDCMFNOAUSQJLUZPSSECTUWP77PCPCI5QJEUL3B5ISRWA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.738749488 (512h17m40.609832523s) [noise=true , address=45.91.116.10:4083 , piece_id=L75J42W77B44T3TPVQHFQ3ED7W6QJOZ3TFHYVRQVQKHMFUALBZZQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.742740798 (512h17m40.613823843s) [noise=true , address=77.90.31.11:4553 , piece_id=VLN6SP5HQ6K6ZH55TTQ3CHQ3K3PK244TATF2IQBSGJUV4GBK4O3Q, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.793210560 (512h17m40.664293595s) [noise=true , address=45.91.116.13:4156 , piece_id=K3NQVWXVO5GFWP6RGEZTRSFEGRDSKGDR52XZBD2XAXQGXO3MO7VA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: <nil> Dec 26 23:46:40.793254373 (512h17m40.664337409s) [seg_pos={0 0}, upload=449528]: Segment reached optimal threshold of 13 pieces. Dec 26 23:46:40.793416341 (512h17m40.664499376s) [noise=true , address=45.91.116.12:4228 , piece_id=GZNWANFP5TGEWUIPKBVBVUPHMCMXEJI7TE6AZUUSSAOM6PSDNMQQ, seg_pos={0 0}, upload=449528]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: expected piece hash; context canceled; EOF Dec 26 23:46:40.793422804 (512h17m40.664505839s) [noise=true , address=45.91.116.12:4228 , piece_id=GZNWANFP5TGEWUIPKBVBVUPHMCMXEJI7TE6AZUUSSAOM6PSDNMQQ, seg_pos={0 0}, upload=449528]: Long tail cancellation due to optimal threshold Dec 26 23:46:40.793665436 (512h17m40.664748471s) [noise=true , address=45.91.116.6:4975 , piece_id=QCSBAVHPFGU2R7NUVOORR76UY3CQYR2TLTMKP72AKEI3QZ7P5OPA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Dec 26 23:46:40.793673451 (512h17m40.664756497s) [noise=true , address=45.91.116.6:4975 , piece_id=QCSBAVHPFGU2R7NUVOORR76UY3CQYR2TLTMKP72AKEI3QZ7P5OPA, seg_pos={0 0}, upload=449528]: Long tail cancellation due to optimal threshold Dec 26 23:46:40.793846872 (512h17m40.664929927s) [noise=true , address=45.91.116.11:4118 , piece_id=ZM4XTNMRDL54XKWIGGMW3PX235TSRCX4USMOD5TYLZ7HXR6LOKOA, seg_pos={0 0}, upload=449528]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: write tcp 188.166.225.170:54658->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:54658->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54658->45.91.116.11:4118: use of closed network connection Dec 26 23:46:40.793854276 (512h17m40.664937311s) [noise=true , address=45.91.116.11:4118 , piece_id=ZM4XTNMRDL54XKWIGGMW3PX235TSRCX4USMOD5TYLZ7HXR6LOKOA, seg_pos={0 0}, upload=449528]: Long tail cancellation due to optimal threshold Dec 26 23:46:40.793869163 (512h17m40.664952199s) [seg_pos={0 0}, upload=449528]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:46:40.793929839 (512h17m40.665012884s) [upload=449528]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:46:40.793933646 (512h17m40.665016681s) [upload=449528]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:46:41.092895268 (512h17m40.963978343s) [upload=449528]: Done uploading segments... Dec 26 23:46:41.092923722 (512h17m40.964006767s) [upload=449528]: Upload finished. err: <nil> Duration: 30.00265492s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.341447603s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:47:04 UTC) orbiter warning: context deadline exceeded Duration: 6.341466639s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..................X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX..] duration: 5.88704444s trace: https://jaeger.dev.storj.io/trace/212569b0e8844898 FAIL (2025-12-26 23:46:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, Piece ID: Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA) context deadline exceeded ======LOGS====== oading segments... Dec 26 23:45:16.426681830 (512h16m16.297764866s) [upload=449506]: Got next segment. Inline: false Dec 26 23:45:16.426708562 (512h16m16.297791597s) [upload=449506]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:45:16.426713531 (512h16m16.297796566s) [upload=449506]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:45:16.677069709 (512h16m16.548152744s) [seg_pos={0 0}, upload=449506]: Begin upload segment... Dec 26 23:45:16.677132048 (512h16m16.548215083s) [seg_pos={0 0}, upload=449506]: Done begin upload segment. Dec 26 23:45:16.677168818 (512h16m16.548251853s) [noise=true , address=45.91.116.6:4973 , piece_id=5IF2OFPIFYKTB5YJB6IZWRXYW6XCDQBRTMAGK7MGNM2DMGXRUYDQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.677208103 (512h16m16.548291138s) [noise=true , address=45.91.116.12:4228 , piece_id=D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.677702284 (512h16m16.548785319s) [noise=true , address=145.79.227.13:4804 , piece_id=YLAUKRCGELEGLEMITJHN3Y4U5PUP6IOLF4DFUQD4HSJFK2HYCQ2A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678093769 (512h16m16.549176804s) [noise=true , address=77.90.31.14:4661 , piece_id=SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678410402 (512h16m16.549493437s) [noise=true , address=145.79.227.52:4445 , piece_id=OTXLTDAWA7S6Q24FL53DAG7PA6W23PPGOIP7D63UHVGSFHUSM54A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678700113 (512h16m16.549783148s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679141714 (512h16m16.550224749s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679446315 (512h16m16.55052935s) [noise=true , address=45.91.116.5:4298 , piece_id=SGQTESPLC37Q3NEYF36TE2EONGA72SNZBH66VRBLTJTYVHHTAJ4Q, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679465300 (512h16m16.550548335s) [noise=true , address=77.90.31.10:4517 , piece_id=Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679812582 (512h16m16.550895617s) [noise=true , address=77.90.31.17:4773 , piece_id=PUO7N2MQRA2DNWZOOHS4ZX57BWJ6HPVFOW43U3XGOPMWHRG2GMOA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680151257 (512h16m16.551234302s) [noise=true , address=145.79.227.51:4411 , piece_id=HZJVZCCOIWLOFU6UTSN5RJUUZKMZFHQ6RGREVFLJCEMW4XINDVTA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680462710 (512h16m16.551545755s) [noise=true , address=77.90.31.21:4877 , piece_id=FF5Y57D7M5TKPOPBJ6TWEBANEJZTZU2P7QYM7MTPKDK22GPSJYQQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680762730 (512h16m16.551845766s) [noise=true , address=145.79.227.16:4910 , piece_id=3XPXPVCUWP5U64ZWRD5Y7VKOALFHTHFPMYEN44UKXYHCQLII4W6A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.681093340 (512h16m16.552176376s) [noise=true , address=145.79.227.45:4229 , piece_id=GKL3TGL4IVCWDJB5TCV3IIXMQURSIZPTPCDVTQ3QNJ4LBMKEFTTA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.681439870 (512h16m16.552522905s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.690933969 (512h16m16.562017004s) [noise=true , address=45.91.116.8:4045 , piece_id=65TLU7EXGR76V3CSHNZWWKFQVQPA2MWQWFAY7CURAN7CAKKYW7WA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:18.411623285 (512h16m18.28270633s) [upload=449506]: Next returned nil segment Dec 26 23:45:18.411652140 (512h16m18.282735185s) [upload=449506]: Waiting for error group managing segments... Dec 26 23:45:18.685740048 (512h16m18.556823083s) [noise=true , address=145.79.227.13:4804 , piece_id=YLAUKRCGELEGLEMITJHN3Y4U5PUP6IOLF4DFUQD4HSJFK2HYCQ2A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.699912443 (512h16m18.570995489s) [noise=true , address=145.79.227.51:4411 , piece_id=HZJVZCCOIWLOFU6UTSN5RJUUZKMZFHQ6RGREVFLJCEMW4XINDVTA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.700425521 (512h16m18.571508566s) [noise=true , address=145.79.227.52:4445 , piece_id=OTXLTDAWA7S6Q24FL53DAG7PA6W23PPGOIP7D63UHVGSFHUSM54A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.703128458 (512h16m18.574211503s) [noise=true , address=145.79.227.16:4910 , piece_id=3XPXPVCUWP5U64ZWRD5Y7VKOALFHTHFPMYEN44UKXYHCQLII4W6A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.720650292 (512h16m18.591733337s) [noise=true , address=145.79.227.45:4229 , piece_id=GKL3TGL4IVCWDJB5TCV3IIXMQURSIZPTPCDVTQ3QNJ4LBMKEFTTA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.879678788 (512h16m18.750761823s) [noise=true , address=77.90.31.14:4661 , piece_id=SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.921650759 (512h16m18.792733804s) [noise=true , address=45.91.116.6:4973 , piece_id=5IF2OFPIFYKTB5YJB6IZWRXYW6XCDQBRTMAGK7MGNM2DMGXRUYDQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.937390089 (512h16m18.808473124s) [noise=true , address=45.91.116.12:4228 , piece_id=D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.949892235 (512h16m18.82097527s) [noise=true , address=77.90.31.10:4517 , piece_id=Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.964177446 (512h16m18.835260491s) [noise=true , address=77.90.31.21:4877 , piece_id=FF5Y57D7M5TKPOPBJ6TWEBANEJZTZU2P7QYM7MTPKDK22GPSJYQQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.995143622 (512h16m18.866226657s) [noise=true , address=45.91.116.5:4298 , piece_id=SGQTESPLC37Q3NEYF36TE2EONGA72SNZBH66VRBLTJTYVHHTAJ4Q, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.006615976 (512h16m18.877699011s) [noise=true , address=77.90.31.17:4773 , piece_id=PUO7N2MQRA2DNWZOOHS4ZX57BWJ6HPVFOW43U3XGOPMWHRG2GMOA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.019410930 (512h16m18.890493965s) [noise=true , address=45.91.116.8:4045 , piece_id=65TLU7EXGR76V3CSHNZWWKFQVQPA2MWQWFAY7CURAN7CAKKYW7WA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.019433483 (512h16m18.890516528s) [seg_pos={0 0}, upload=449506]: Segment reached optimal threshold of 13 pieces. Dec 26 23:45:19.019652379 (512h16m18.890735415s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Dec 26 23:45:19.019660665 (512h16m18.8907437s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019775945 (512h16m18.89085898s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection Dec 26 23:45:19.019781325 (512h16m18.89086436s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019874102 (512h16m18.890957137s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection; write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection Dec 26 23:45:19.019883570 (512h16m18.890966605s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019902405 (512h16m18.89098544s) [seg_pos={0 0}, upload=449506]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:45:19.019972659 (512h16m18.891055704s) [upload=449506]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:45:19.019978430 (512h16m18.891061465s) [upload=449506]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:45:19.380451377 (512h16m19.251534423s) [upload=449506]: Done uploading segments... Dec 26 23:45:19.380470855 (512h16m19.25155389s) [upload=449506]: Upload finished. err: <nil> Duration: 5.887072124s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 255.193328ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/67e64704eddcd2cd FAIL (2025-12-26 23:46:02 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 255.21006ms edge linksharing bgp eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.400695884s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:45:52 UTC) orbiter warning: context deadline exceeded Duration: 6.400722606s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [...................X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX.] duration: 6.026546996s trace: https://jaeger.dev.storj.io/trace/212569b0e8844898 FAIL (2025-12-26 23:45:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, Piece ID: Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA) context deadline exceeded ======LOGS====== oading segments... Dec 26 23:45:16.426681830 (512h16m16.297764866s) [upload=449506]: Got next segment. Inline: false Dec 26 23:45:16.426708562 (512h16m16.297791597s) [upload=449506]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:45:16.426713531 (512h16m16.297796566s) [upload=449506]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:45:16.677069709 (512h16m16.548152744s) [seg_pos={0 0}, upload=449506]: Begin upload segment... Dec 26 23:45:16.677132048 (512h16m16.548215083s) [seg_pos={0 0}, upload=449506]: Done begin upload segment. Dec 26 23:45:16.677168818 (512h16m16.548251853s) [noise=true , address=45.91.116.6:4973 , piece_id=5IF2OFPIFYKTB5YJB6IZWRXYW6XCDQBRTMAGK7MGNM2DMGXRUYDQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.677208103 (512h16m16.548291138s) [noise=true , address=45.91.116.12:4228 , piece_id=D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.677702284 (512h16m16.548785319s) [noise=true , address=145.79.227.13:4804 , piece_id=YLAUKRCGELEGLEMITJHN3Y4U5PUP6IOLF4DFUQD4HSJFK2HYCQ2A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678093769 (512h16m16.549176804s) [noise=true , address=77.90.31.14:4661 , piece_id=SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678410402 (512h16m16.549493437s) [noise=true , address=145.79.227.52:4445 , piece_id=OTXLTDAWA7S6Q24FL53DAG7PA6W23PPGOIP7D63UHVGSFHUSM54A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678700113 (512h16m16.549783148s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679141714 (512h16m16.550224749s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679446315 (512h16m16.55052935s) [noise=true , address=45.91.116.5:4298 , piece_id=SGQTESPLC37Q3NEYF36TE2EONGA72SNZBH66VRBLTJTYVHHTAJ4Q, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679465300 (512h16m16.550548335s) [noise=true , address=77.90.31.10:4517 , piece_id=Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679812582 (512h16m16.550895617s) [noise=true , address=77.90.31.17:4773 , piece_id=PUO7N2MQRA2DNWZOOHS4ZX57BWJ6HPVFOW43U3XGOPMWHRG2GMOA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680151257 (512h16m16.551234302s) [noise=true , address=145.79.227.51:4411 , piece_id=HZJVZCCOIWLOFU6UTSN5RJUUZKMZFHQ6RGREVFLJCEMW4XINDVTA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680462710 (512h16m16.551545755s) [noise=true , address=77.90.31.21:4877 , piece_id=FF5Y57D7M5TKPOPBJ6TWEBANEJZTZU2P7QYM7MTPKDK22GPSJYQQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680762730 (512h16m16.551845766s) [noise=true , address=145.79.227.16:4910 , piece_id=3XPXPVCUWP5U64ZWRD5Y7VKOALFHTHFPMYEN44UKXYHCQLII4W6A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.681093340 (512h16m16.552176376s) [noise=true , address=145.79.227.45:4229 , piece_id=GKL3TGL4IVCWDJB5TCV3IIXMQURSIZPTPCDVTQ3QNJ4LBMKEFTTA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.681439870 (512h16m16.552522905s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.690933969 (512h16m16.562017004s) [noise=true , address=45.91.116.8:4045 , piece_id=65TLU7EXGR76V3CSHNZWWKFQVQPA2MWQWFAY7CURAN7CAKKYW7WA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:18.411623285 (512h16m18.28270633s) [upload=449506]: Next returned nil segment Dec 26 23:45:18.411652140 (512h16m18.282735185s) [upload=449506]: Waiting for error group managing segments... Dec 26 23:45:18.685740048 (512h16m18.556823083s) [noise=true , address=145.79.227.13:4804 , piece_id=YLAUKRCGELEGLEMITJHN3Y4U5PUP6IOLF4DFUQD4HSJFK2HYCQ2A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.699912443 (512h16m18.570995489s) [noise=true , address=145.79.227.51:4411 , piece_id=HZJVZCCOIWLOFU6UTSN5RJUUZKMZFHQ6RGREVFLJCEMW4XINDVTA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.700425521 (512h16m18.571508566s) [noise=true , address=145.79.227.52:4445 , piece_id=OTXLTDAWA7S6Q24FL53DAG7PA6W23PPGOIP7D63UHVGSFHUSM54A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.703128458 (512h16m18.574211503s) [noise=true , address=145.79.227.16:4910 , piece_id=3XPXPVCUWP5U64ZWRD5Y7VKOALFHTHFPMYEN44UKXYHCQLII4W6A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.720650292 (512h16m18.591733337s) [noise=true , address=145.79.227.45:4229 , piece_id=GKL3TGL4IVCWDJB5TCV3IIXMQURSIZPTPCDVTQ3QNJ4LBMKEFTTA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.879678788 (512h16m18.750761823s) [noise=true , address=77.90.31.14:4661 , piece_id=SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.921650759 (512h16m18.792733804s) [noise=true , address=45.91.116.6:4973 , piece_id=5IF2OFPIFYKTB5YJB6IZWRXYW6XCDQBRTMAGK7MGNM2DMGXRUYDQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.937390089 (512h16m18.808473124s) [noise=true , address=45.91.116.12:4228 , piece_id=D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.949892235 (512h16m18.82097527s) [noise=true , address=77.90.31.10:4517 , piece_id=Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.964177446 (512h16m18.835260491s) [noise=true , address=77.90.31.21:4877 , piece_id=FF5Y57D7M5TKPOPBJ6TWEBANEJZTZU2P7QYM7MTPKDK22GPSJYQQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.995143622 (512h16m18.866226657s) [noise=true , address=45.91.116.5:4298 , piece_id=SGQTESPLC37Q3NEYF36TE2EONGA72SNZBH66VRBLTJTYVHHTAJ4Q, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.006615976 (512h16m18.877699011s) [noise=true , address=77.90.31.17:4773 , piece_id=PUO7N2MQRA2DNWZOOHS4ZX57BWJ6HPVFOW43U3XGOPMWHRG2GMOA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.019410930 (512h16m18.890493965s) [noise=true , address=45.91.116.8:4045 , piece_id=65TLU7EXGR76V3CSHNZWWKFQVQPA2MWQWFAY7CURAN7CAKKYW7WA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.019433483 (512h16m18.890516528s) [seg_pos={0 0}, upload=449506]: Segment reached optimal threshold of 13 pieces. Dec 26 23:45:19.019652379 (512h16m18.890735415s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Dec 26 23:45:19.019660665 (512h16m18.8907437s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019775945 (512h16m18.89085898s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection Dec 26 23:45:19.019781325 (512h16m18.89086436s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019874102 (512h16m18.890957137s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection; write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection Dec 26 23:45:19.019883570 (512h16m18.890966605s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019902405 (512h16m18.89098544s) [seg_pos={0 0}, upload=449506]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:45:19.019972659 (512h16m18.891055704s) [upload=449506]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:45:19.019978430 (512h16m18.891061465s) [upload=449506]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:45:19.380451377 (512h16m19.251534423s) [upload=449506]: Done uploading segments... Dec 26 23:45:19.380470855 (512h16m19.25155389s) [upload=449506]: Upload finished. err: <nil> Duration: 6.026559589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [....................X.X.X.X.XX....X.X.X.......X..XX.XX..XXXX] duration: 30.003924832s trace: https://jaeger.dev.storj.io/trace/212569b0e8844898 FAIL (2025-12-26 23:45:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, Piece ID: Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ) context deadline exceeded; piecestore: (Node ID: 12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA, Piece ID: D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA) context deadline exceeded ======LOGS====== oading segments... Dec 26 23:45:16.426681830 (512h16m16.297764866s) [upload=449506]: Got next segment. Inline: false Dec 26 23:45:16.426708562 (512h16m16.297791597s) [upload=449506]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:45:16.426713531 (512h16m16.297796566s) [upload=449506]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:45:16.677069709 (512h16m16.548152744s) [seg_pos={0 0}, upload=449506]: Begin upload segment... Dec 26 23:45:16.677132048 (512h16m16.548215083s) [seg_pos={0 0}, upload=449506]: Done begin upload segment. Dec 26 23:45:16.677168818 (512h16m16.548251853s) [noise=true , address=45.91.116.6:4973 , piece_id=5IF2OFPIFYKTB5YJB6IZWRXYW6XCDQBRTMAGK7MGNM2DMGXRUYDQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.677208103 (512h16m16.548291138s) [noise=true , address=45.91.116.12:4228 , piece_id=D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.677702284 (512h16m16.548785319s) [noise=true , address=145.79.227.13:4804 , piece_id=YLAUKRCGELEGLEMITJHN3Y4U5PUP6IOLF4DFUQD4HSJFK2HYCQ2A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678093769 (512h16m16.549176804s) [noise=true , address=77.90.31.14:4661 , piece_id=SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678410402 (512h16m16.549493437s) [noise=true , address=145.79.227.52:4445 , piece_id=OTXLTDAWA7S6Q24FL53DAG7PA6W23PPGOIP7D63UHVGSFHUSM54A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.678700113 (512h16m16.549783148s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679141714 (512h16m16.550224749s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679446315 (512h16m16.55052935s) [noise=true , address=45.91.116.5:4298 , piece_id=SGQTESPLC37Q3NEYF36TE2EONGA72SNZBH66VRBLTJTYVHHTAJ4Q, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679465300 (512h16m16.550548335s) [noise=true , address=77.90.31.10:4517 , piece_id=Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.679812582 (512h16m16.550895617s) [noise=true , address=77.90.31.17:4773 , piece_id=PUO7N2MQRA2DNWZOOHS4ZX57BWJ6HPVFOW43U3XGOPMWHRG2GMOA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680151257 (512h16m16.551234302s) [noise=true , address=145.79.227.51:4411 , piece_id=HZJVZCCOIWLOFU6UTSN5RJUUZKMZFHQ6RGREVFLJCEMW4XINDVTA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680462710 (512h16m16.551545755s) [noise=true , address=77.90.31.21:4877 , piece_id=FF5Y57D7M5TKPOPBJ6TWEBANEJZTZU2P7QYM7MTPKDK22GPSJYQQ, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.680762730 (512h16m16.551845766s) [noise=true , address=145.79.227.16:4910 , piece_id=3XPXPVCUWP5U64ZWRD5Y7VKOALFHTHFPMYEN44UKXYHCQLII4W6A, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.681093340 (512h16m16.552176376s) [noise=true , address=145.79.227.45:4229 , piece_id=GKL3TGL4IVCWDJB5TCV3IIXMQURSIZPTPCDVTQ3QNJ4LBMKEFTTA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.681439870 (512h16m16.552522905s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:16.690933969 (512h16m16.562017004s) [noise=true , address=45.91.116.8:4045 , piece_id=65TLU7EXGR76V3CSHNZWWKFQVQPA2MWQWFAY7CURAN7CAKKYW7WA, seg_pos={0 0}, upload=449506]: Uploading piece... Dec 26 23:45:18.411623285 (512h16m18.28270633s) [upload=449506]: Next returned nil segment Dec 26 23:45:18.411652140 (512h16m18.282735185s) [upload=449506]: Waiting for error group managing segments... Dec 26 23:45:18.685740048 (512h16m18.556823083s) [noise=true , address=145.79.227.13:4804 , piece_id=YLAUKRCGELEGLEMITJHN3Y4U5PUP6IOLF4DFUQD4HSJFK2HYCQ2A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.699912443 (512h16m18.570995489s) [noise=true , address=145.79.227.51:4411 , piece_id=HZJVZCCOIWLOFU6UTSN5RJUUZKMZFHQ6RGREVFLJCEMW4XINDVTA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.700425521 (512h16m18.571508566s) [noise=true , address=145.79.227.52:4445 , piece_id=OTXLTDAWA7S6Q24FL53DAG7PA6W23PPGOIP7D63UHVGSFHUSM54A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.703128458 (512h16m18.574211503s) [noise=true , address=145.79.227.16:4910 , piece_id=3XPXPVCUWP5U64ZWRD5Y7VKOALFHTHFPMYEN44UKXYHCQLII4W6A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.720650292 (512h16m18.591733337s) [noise=true , address=145.79.227.45:4229 , piece_id=GKL3TGL4IVCWDJB5TCV3IIXMQURSIZPTPCDVTQ3QNJ4LBMKEFTTA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.879678788 (512h16m18.750761823s) [noise=true , address=77.90.31.14:4661 , piece_id=SHMC3L3DMIQKV3KRN3DNRVW5N2MVRMPOY3APLHQ72FCGPEPYLNCA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.921650759 (512h16m18.792733804s) [noise=true , address=45.91.116.6:4973 , piece_id=5IF2OFPIFYKTB5YJB6IZWRXYW6XCDQBRTMAGK7MGNM2DMGXRUYDQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.937390089 (512h16m18.808473124s) [noise=true , address=45.91.116.12:4228 , piece_id=D7EW5RFC35EPNDCWV4BTCLJJFV53PT2VM2IJWGZUH7U6ZQLMJP2A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.949892235 (512h16m18.82097527s) [noise=true , address=77.90.31.10:4517 , piece_id=Z6PTF6ZPPX2XYFDQMMQRWAVKKHZ57QV63IQVBT4QMVDKFYBCDFEQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.964177446 (512h16m18.835260491s) [noise=true , address=77.90.31.21:4877 , piece_id=FF5Y57D7M5TKPOPBJ6TWEBANEJZTZU2P7QYM7MTPKDK22GPSJYQQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:18.995143622 (512h16m18.866226657s) [noise=true , address=45.91.116.5:4298 , piece_id=SGQTESPLC37Q3NEYF36TE2EONGA72SNZBH66VRBLTJTYVHHTAJ4Q, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.006615976 (512h16m18.877699011s) [noise=true , address=77.90.31.17:4773 , piece_id=PUO7N2MQRA2DNWZOOHS4ZX57BWJ6HPVFOW43U3XGOPMWHRG2GMOA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.019410930 (512h16m18.890493965s) [noise=true , address=45.91.116.8:4045 , piece_id=65TLU7EXGR76V3CSHNZWWKFQVQPA2MWQWFAY7CURAN7CAKKYW7WA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: <nil> Dec 26 23:45:19.019433483 (512h16m18.890516528s) [seg_pos={0 0}, upload=449506]: Segment reached optimal threshold of 13 pieces. Dec 26 23:45:19.019652379 (512h16m18.890735415s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Dec 26 23:45:19.019660665 (512h16m18.8907437s) [noise=true , address=45.91.116.11:4119 , piece_id=IXHQASPDWS4A56PHYYDY3BUBDCE4J3UPYYKK5LRDGWJU2WZGDN4A, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019775945 (512h16m18.89085898s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59450->45.91.116.9:4263: use of closed network connection Dec 26 23:45:19.019781325 (512h16m18.89086436s) [noise=true , address=45.91.116.9:4263 , piece_id=OPWNAGEESMGNNVCALUC4ABIO4L3EODYB5RBMCWZRBGJNQA2WCNWA, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019874102 (512h16m18.890957137s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DqyGgLKeQEoD6e4TLmUJ6kQYPsZkKe9WNJf3HxWn3erAargYk): protocol: write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection; write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43304->77.90.31.16:4736: use of closed network connection Dec 26 23:45:19.019883570 (512h16m18.890966605s) [noise=true , address=77.90.31.16:4736 , piece_id=L7D44M7YML4Z3N3WKTGQMNFUQDAHMEBFQ7KPJMSYC7D4KMVEBLYQ, seg_pos={0 0}, upload=449506]: Long tail cancellation due to optimal threshold Dec 26 23:45:19.019902405 (512h16m18.89098544s) [seg_pos={0 0}, upload=449506]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:45:19.019972659 (512h16m18.891055704s) [upload=449506]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:45:19.019978430 (512h16m18.891061465s) [upload=449506]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:45:19.380451377 (512h16m19.251534423s) [upload=449506]: Done uploading segments... Dec 26 23:45:19.380470855 (512h16m19.25155389s) [upload=449506]: Upload finished. err: <nil> Duration: 30.003948428s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.357500772s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:45:13 UTC) orbiter warning: context deadline exceeded Duration: 6.357528787s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.....................X.X.X.X.XX....X.X.X.......X..XX.XX..XXX] duration: 30.002594645s trace: https://jaeger.dev.storj.io/trace/633819b658043a7c FAIL (2025-12-26 23:44:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, Piece ID: 3YIJW2D4WOOKXEMYFNBZZHZ7JZ4L6UE4Z6KWOBUI47XMCMUVOYXQ) context deadline exceeded; piecestore: (Node ID: 129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H, Piece ID: TFFXDHUZJJGEQPGFGXQL23VO76YYDVJF4Z6BNO7JH7ATNLMJ5OHQ) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: EUHVMUNB6DM7WI7RNCUWEOMSFRLRHUFE6VWGMWEDOFXJZQ6L6SFA) context deadline exceeded; piecestore: (Node ID: 1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, Piece ID: 5YTUGW7N32ERBREYOBEIQW36FHBLZJBINKJR7LRBX2KCWJ37EMRQ) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: UEOILIQJBHCI3LJNREMS5TVGXOFVDVYRHSSJVNUK4EILLKJYXY7A) context deadline exceeded ======LOGS====== Dec 26 23:44:05.135823308 (512h15m5.006906353s) [upload=449490]: Starting upload Dec 26 23:44:05.135898271 (512h15m5.006981316s) [upload=449490]: Done starting upload Dec 26 23:44:05.135996558 (512h15m5.007079603s) [upload=449490]: Uploading segments... Dec 26 23:44:05.136162604 (512h15m5.007245649s) [upload=449490]: Got next segment. Inline: false Dec 26 23:44:05.136200897 (512h15m5.007283932s) [upload=449490]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:44:05.136208892 (512h15m5.007291937s) [upload=449490]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:44:05.399843414 (512h15m5.270926449s) [seg_pos={0 0}, upload=449490]: Begin upload segment... Dec 26 23:44:05.399897457 (512h15m5.270980492s) [seg_pos={0 0}, upload=449490]: Done begin upload segment. Dec 26 23:44:05.399923517 (512h15m5.271006562s) [noise=true , address=145.79.227.47:4267 , piece_id=GMEJ7GSPLISN5LMSSPAXDWJSDO26UNSMBHC4HBSEU5XQVK3K7T3A, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.399978251 (512h15m5.271061296s) [noise=true , address=145.79.227.17:4972 , piece_id=DYCE6DIRYOWV5RV5PGR2UPKJOYU4W3SULZB6U7FS7GHX7WOQ42UA, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.400537826 (512h15m5.271620871s) [noise=true , address=145.79.227.15:4876 , piece_id=EUHVMUNB6DM7WI7RNCUWEOMSFRLRHUFE6VWGMWEDOFXJZQ6L6SFA, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.400880780 (512h15m5.271963815s) [noise=true , address=45.91.116.9:4263 , piece_id=CEX5OHLGSOIIZIOWWSKWDB6A4RBFBSZN5J4CABWNRWIE37TABWFA, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.400904004 (512h15m5.271987039s) [noise=true , address=145.79.227.50:4374 , piece_id=2P3FSYG7IB3LOBBUNC6FXCNVDUD7W4FVY27XIOO565BE75UGYS3A, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.401201009 (512h15m5.272284044s) [noise=true , address=145.79.227.41:4087 , piece_id=UC3HLEJMFLX66Y6N2CSEEB7PM7VTL5CONB3BQ2CAI6IP7BILHSJA, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.401307472 (512h15m5.272390507s) [noise=true , address=45.91.116.4:4940 , piece_id=ZC7FDCXKUCAMWRLRDL4TZNZX5KR7PSASZTRCJ36ODUD6C7EWX33A, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.401460514 (512h15m5.272543549s) [noise=true , address=145.79.227.46:4602 , piece_id=5YTUGW7N32ERBREYOBEIQW36FHBLZJBINKJR7LRBX2KCWJ37EMRQ, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.401650946 (512h15m5.272733981s) [noise=true , address=145.79.227.52:4446 , piece_id=Y4B7LZVXWUM4UWSSCCCTAM7IWXHTW2WXRJQR4PGWOA2LBQF7XIHQ, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.401743773 (512h15m5.272826818s) [noise=true , address=45.91.116.13:4154 , piece_id=UEOILIQJBHCI3LJNREMS5TVGXOFVDVYRHSSJVNUK4EILLKJYXY7A, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.402035648 (512h15m5.273118684s) [noise=true , address=45.91.116.5:4298 , piece_id=MHYG34EE52KJLBO6R5VRUQLUBOZANN2REZZ4QHXYA3XHOUQ24A4Q, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.402299942 (512h15m5.273382987s) [noise=true , address=77.90.31.14:4662 , piece_id=3YIJW2D4WOOKXEMYFNBZZHZ7JZ4L6UE4Z6KWOBUI47XMCMUVOYXQ, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.402417866 (512h15m5.273500901s) [noise=true , address=45.91.116.8:4046 , piece_id=TFFXDHUZJJGEQPGFGXQL23VO76YYDVJF4Z6BNO7JH7ATNLMJ5OHQ, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.402622506 (512h15m5.273705541s) [noise=true , address=145.79.227.37:4565 , piece_id=VPY67VHSL3UE7K5TLEA4CSUHNKMKHBQGFBH6UG7ETVOYHV5GKP3Q, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.402756782 (512h15m5.273839827s) [noise=true , address=145.79.227.39:4013 , piece_id=UWAGXEAAJCLTB3ZDMKQCYLN4SQCFOQI2J22UQL2PXTHIW4FGBFTA, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:05.402963165 (512h15m5.27404621s) [noise=true , address=145.79.227.13:4804 , piece_id=E3TJX5V5EEUYQCSSVH7TZSPH7MEH4RBXGQKRBAY3VBHUEK6JKFDQ, seg_pos={0 0}, upload=449490]: Uploading piece... Dec 26 23:44:07.090525082 (512h15m6.961608117s) [upload=449490]: Next returned nil segment Dec 26 23:44:07.090595476 (512h15m6.961678531s) [upload=449490]: Waiting for error group managing segments... Dec 26 23:44:07.407729891 (512h15m7.278812926s) [noise=true , address=145.79.227.46:4602 , piece_id=5YTUGW7N32ERBREYOBEIQW36FHBLZJBINKJR7LRBX2KCWJ37EMRQ, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.410080747 (512h15m7.281163792s) [noise=true , address=145.79.227.37:4565 , piece_id=VPY67VHSL3UE7K5TLEA4CSUHNKMKHBQGFBH6UG7ETVOYHV5GKP3Q, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.410357675 (512h15m7.28144072s) [noise=true , address=145.79.227.39:4013 , piece_id=UWAGXEAAJCLTB3ZDMKQCYLN4SQCFOQI2J22UQL2PXTHIW4FGBFTA, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.411429956 (512h15m7.282512991s) [noise=true , address=145.79.227.50:4374 , piece_id=2P3FSYG7IB3LOBBUNC6FXCNVDUD7W4FVY27XIOO565BE75UGYS3A, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.423106970 (512h15m7.294190015s) [noise=true , address=145.79.227.15:4876 , piece_id=EUHVMUNB6DM7WI7RNCUWEOMSFRLRHUFE6VWGMWEDOFXJZQ6L6SFA, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.423250293 (512h15m7.294333328s) [noise=true , address=145.79.227.41:4087 , piece_id=UC3HLEJMFLX66Y6N2CSEEB7PM7VTL5CONB3BQ2CAI6IP7BILHSJA, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.426989324 (512h15m7.298072359s) [noise=true , address=145.79.227.47:4267 , piece_id=GMEJ7GSPLISN5LMSSPAXDWJSDO26UNSMBHC4HBSEU5XQVK3K7T3A, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.427905227 (512h15m7.298988262s) [noise=true , address=145.79.227.17:4972 , piece_id=DYCE6DIRYOWV5RV5PGR2UPKJOYU4W3SULZB6U7FS7GHX7WOQ42UA, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.430827342 (512h15m7.301910387s) [noise=true , address=145.79.227.52:4446 , piece_id=Y4B7LZVXWUM4UWSSCCCTAM7IWXHTW2WXRJQR4PGWOA2LBQF7XIHQ, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.438114007 (512h15m7.309197042s) [noise=true , address=145.79.227.13:4804 , piece_id=E3TJX5V5EEUYQCSSVH7TZSPH7MEH4RBXGQKRBAY3VBHUEK6JKFDQ, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.638700566 (512h15m7.509783611s) [noise=true , address=45.91.116.13:4154 , piece_id=UEOILIQJBHCI3LJNREMS5TVGXOFVDVYRHSSJVNUK4EILLKJYXY7A, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.664708349 (512h15m7.535791384s) [noise=true , address=45.91.116.8:4046 , piece_id=TFFXDHUZJJGEQPGFGXQL23VO76YYDVJF4Z6BNO7JH7ATNLMJ5OHQ, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.690475123 (512h15m7.561558168s) [noise=true , address=77.90.31.14:4662 , piece_id=3YIJW2D4WOOKXEMYFNBZZHZ7JZ4L6UE4Z6KWOBUI47XMCMUVOYXQ, seg_pos={0 0}, upload=449490]: Done uploading piece. err: <nil> Dec 26 23:44:07.690496704 (512h15m7.561579739s) [seg_pos={0 0}, upload=449490]: Segment reached optimal threshold of 13 pieces. Dec 26 23:44:07.690696234 (512h15m7.561779289s) [noise=true , address=45.91.116.5:4298 , piece_id=MHYG34EE52KJLBO6R5VRUQLUBOZANN2REZZ4QHXYA3XHOUQ24A4Q, seg_pos={0 0}, upload=449490]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: write tcp 188.166.225.170:51656->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:51656->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51656->45.91.116.5:4298: use of closed network connection Dec 26 23:44:07.690704881 (512h15m7.561787926s) [noise=true , address=45.91.116.5:4298 , piece_id=MHYG34EE52KJLBO6R5VRUQLUBOZANN2REZZ4QHXYA3XHOUQ24A4Q, seg_pos={0 0}, upload=449490]: Long tail cancellation due to optimal threshold Dec 26 23:44:07.690994452 (512h15m7.562077497s) [noise=true , address=45.91.116.9:4263 , piece_id=CEX5OHLGSOIIZIOWWSKWDB6A4RBFBSZN5J4CABWNRWIE37TABWFA, seg_pos={0 0}, upload=449490]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Dec 26 23:44:07.691056430 (512h15m7.562139465s) [noise=true , address=45.91.116.9:4263 , piece_id=CEX5OHLGSOIIZIOWWSKWDB6A4RBFBSZN5J4CABWNRWIE37TABWFA, seg_pos={0 0}, upload=449490]: Long tail cancellation due to optimal threshold Dec 26 23:44:07.691168764 (512h15m7.562251809s) [noise=true , address=45.91.116.4:4940 , piece_id=ZC7FDCXKUCAMWRLRDL4TZNZX5KR7PSASZTRCJ36ODUD6C7EWX33A, seg_pos={0 0}, upload=449490]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: expected piece hash; context canceled; EOF Dec 26 23:44:07.691180325 (512h15m7.562263361s) [noise=true , address=45.91.116.4:4940 , piece_id=ZC7FDCXKUCAMWRLRDL4TZNZX5KR7PSASZTRCJ36ODUD6C7EWX33A, seg_pos={0 0}, upload=449490]: Long tail cancellation due to optimal threshold Dec 26 23:44:07.691278743 (512h15m7.562361788s) [seg_pos={0 0}, upload=449490]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:44:07.691438508 (512h15m7.562521543s) [upload=449490]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:44:07.691446362 (512h15m7.562529407s) [upload=449490]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:44:08.047216125 (512h15m7.91829917s) [upload=449490]: Done uploading segments... Dec 26 23:44:08.047234480 (512h15m7.918317525s) [upload=449490]: Upload finished. err: <nil> Duration: 30.002608622s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.279836967s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:44:32 UTC) orbiter warning: context deadline exceeded Duration: 6.280391985s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [......................X.X.X.X.XX....X.X.X.......X..XX.XX..XX] duration: 30.003889426s trace: https://jaeger.dev.storj.io/trace/25d581ad3c174638 FAIL (2025-12-26 23:44:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, Piece ID: B4FFIAL35KZBZDYMWQOALEDQ3EKNXY2VFNNGTDFRRFJIYN6R7NDA) context deadline exceeded; piecestore: (Node ID: 1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8, Piece ID: QJUJYFEBOW3IC2GXJ6WC2EBAB42QYD5GO7HENRIZPJMM3M7XYPXA) context deadline exceeded ======LOGS====== 26 23:43:34.886843009 (512h14m34.757926044s) [upload=449478]: Starting upload Dec 26 23:43:34.886893605 (512h14m34.75797664s) [upload=449478]: Done starting upload Dec 26 23:43:34.886967606 (512h14m34.758050642s) [upload=449478]: Uploading segments... Dec 26 23:43:34.887155434 (512h14m34.758238499s) [upload=449478]: Got next segment. Inline: false Dec 26 23:43:34.887194007 (512h14m34.758277042s) [upload=449478]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:43:34.887199658 (512h14m34.758282693s) [upload=449478]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:43:35.140009438 (512h14m35.011092473s) [seg_pos={0 0}, upload=449478]: Begin upload segment... Dec 26 23:43:35.140061496 (512h14m35.011144531s) [seg_pos={0 0}, upload=449478]: Done begin upload segment. Dec 26 23:43:35.140127552 (512h14m35.011210587s) [noise=true , address=45.91.116.11:4120 , piece_id=QJUJYFEBOW3IC2GXJ6WC2EBAB42QYD5GO7HENRIZPJMM3M7XYPXA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.140135397 (512h14m35.011218432s) [noise=true , address=145.79.227.29:4942 , piece_id=TWKEUY2AZYLT66SAL3S6LQOHJL5KY54VDMX4TPPOFBYBNJU73OCA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.140928147 (512h14m35.012011182s) [noise=true , address=145.79.227.42:4122 , piece_id=66QOX42VKCTIMLF75GBEBW6VMZZ334UKAWAKW4P4LNAZS7FPU4AQ, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.141259007 (512h14m35.012342052s) [noise=true , address=77.90.31.18:4805 , piece_id=XF7P3HKRNVXZDNZDLC44DL3YGELLQLHX6I7RRIC3SCXROO2V7U4A, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.142252008 (512h14m35.013335043s) [noise=true , address=145.79.227.46:4602 , piece_id=5NWCLH2LJEBXDY45ZY5U4F5BPIZXPPXUH6DFR37IVLAA7RF7E6QA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.142920530 (512h14m35.014003565s) [noise=true , address=77.90.31.15:4697 , piece_id=REFVB35UAV25YSHFM3DAGSRVY3NFHUCUL5JWYCY6R6WTWYUEUVLA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.142984783 (512h14m35.014067828s) [noise=true , address=45.91.116.9:4261 , piece_id=D3IDG4BUI7RDUI53PXQQH4NSBLNRUVNN4XOIDJYPT3MV5RJJQ5IA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.143469917 (512h14m35.014552952s) [noise=true , address=145.79.227.40:4049 , piece_id=3IE65OPLHMMQKH5CMNND2W7QV4OMK7WTRP2ICWDS77OK23CJIEKA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.143845602 (512h14m35.014928647s) [noise=true , address=45.91.116.14:4189 , piece_id=5Q4HPMF35MDOX5GRDFAMPCFRNAG4VIMT2RADQPGIHLWXBPAWUO3Q, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.144203483 (512h14m35.015286518s) [noise=true , address=145.79.227.45:4230 , piece_id=SJWKAKYW362ICPXQXNQFUTE4INHVS7COZ46I26OKGPILKW4IGLGQ, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.144541447 (512h14m35.015624482s) [noise=true , address=145.79.227.2:4115 , piece_id=JFKOTVWUVHFPPYUJX3GGAELRCRWBNRLQRER5TCVEMDG2HADE5GUA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.144883679 (512h14m35.015966714s) [noise=true , address=145.79.227.51:4411 , piece_id=KPY6X2BGXQSTYR4Q54H2E2KYKDZIOXQI7TCDZH36Y5NEJJIAN3SQ, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.146596490 (512h14m35.017679525s) [noise=true , address=145.79.227.15:4876 , piece_id=6QKAJR4D3CX6BKIQNL64LGQAWE354BF47NRXBCIU4YFIRQURSJYQ, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.147590293 (512h14m35.018673328s) [noise=true , address=145.79.227.6:4257 , piece_id=P5OYCVFYZCBNJ47ELG3CEYR7U4EDJ46JYPVGWQYZNCHHSIMOWW6Q, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.147828606 (512h14m35.018911651s) [noise=true , address=45.91.116.13:4153 , piece_id=B4FFIAL35KZBZDYMWQOALEDQ3EKNXY2VFNNGTDFRRFJIYN6R7NDA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:35.149029543 (512h14m35.020112578s) [noise=true , address=45.91.116.7:4011 , piece_id=G65WREW4MV5ITP3DUO7QTHFEQ4MJECFLXLJO7FVHHV5B37YRK5CA, seg_pos={0 0}, upload=449478]: Uploading piece... Dec 26 23:43:36.756890016 (512h14m36.627973051s) [upload=449478]: Next returned nil segment Dec 26 23:43:36.756906127 (512h14m36.627989162s) [upload=449478]: Waiting for error group managing segments... Dec 26 23:43:37.143818028 (512h14m37.014901073s) [noise=true , address=145.79.227.42:4122 , piece_id=66QOX42VKCTIMLF75GBEBW6VMZZ334UKAWAKW4P4LNAZS7FPU4AQ, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.144155060 (512h14m37.015238105s) [noise=true , address=145.79.227.46:4602 , piece_id=5NWCLH2LJEBXDY45ZY5U4F5BPIZXPPXUH6DFR37IVLAA7RF7E6QA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.151177762 (512h14m37.022260807s) [noise=true , address=145.79.227.45:4230 , piece_id=SJWKAKYW362ICPXQXNQFUTE4INHVS7COZ46I26OKGPILKW4IGLGQ, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.156312640 (512h14m37.027395675s) [noise=true , address=145.79.227.15:4876 , piece_id=6QKAJR4D3CX6BKIQNL64LGQAWE354BF47NRXBCIU4YFIRQURSJYQ, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.171106009 (512h14m37.042189044s) [noise=true , address=145.79.227.6:4257 , piece_id=P5OYCVFYZCBNJ47ELG3CEYR7U4EDJ46JYPVGWQYZNCHHSIMOWW6Q, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.184277750 (512h14m37.055360785s) [noise=true , address=145.79.227.29:4942 , piece_id=TWKEUY2AZYLT66SAL3S6LQOHJL5KY54VDMX4TPPOFBYBNJU73OCA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.190078495 (512h14m37.06116153s) [noise=true , address=145.79.227.51:4411 , piece_id=KPY6X2BGXQSTYR4Q54H2E2KYKDZIOXQI7TCDZH36Y5NEJJIAN3SQ, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.337649615 (512h14m37.20873265s) [noise=true , address=77.90.31.18:4805 , piece_id=XF7P3HKRNVXZDNZDLC44DL3YGELLQLHX6I7RRIC3SCXROO2V7U4A, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.376272880 (512h14m37.247355915s) [noise=true , address=45.91.116.9:4261 , piece_id=D3IDG4BUI7RDUI53PXQQH4NSBLNRUVNN4XOIDJYPT3MV5RJJQ5IA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.394591411 (512h14m37.265674446s) [noise=true , address=45.91.116.11:4120 , piece_id=QJUJYFEBOW3IC2GXJ6WC2EBAB42QYD5GO7HENRIZPJMM3M7XYPXA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.396147605 (512h14m37.26723064s) [noise=true , address=45.91.116.13:4153 , piece_id=B4FFIAL35KZBZDYMWQOALEDQ3EKNXY2VFNNGTDFRRFJIYN6R7NDA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.433599759 (512h14m37.304682804s) [noise=true , address=145.79.227.40:4049 , piece_id=3IE65OPLHMMQKH5CMNND2W7QV4OMK7WTRP2ICWDS77OK23CJIEKA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.524240745 (512h14m37.39532379s) [noise=true , address=45.91.116.7:4011 , piece_id=G65WREW4MV5ITP3DUO7QTHFEQ4MJECFLXLJO7FVHHV5B37YRK5CA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: <nil> Dec 26 23:43:37.524264048 (512h14m37.395347093s) [seg_pos={0 0}, upload=449478]: Segment reached optimal threshold of 13 pieces. Dec 26 23:43:37.524515487 (512h14m37.395598532s) [noise=true , address=77.90.31.15:4697 , piece_id=REFVB35UAV25YSHFM3DAGSRVY3NFHUCUL5JWYCY6R6WTWYUEUVLA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Dec 26 23:43:37.524525166 (512h14m37.395608201s) [noise=true , address=77.90.31.15:4697 , piece_id=REFVB35UAV25YSHFM3DAGSRVY3NFHUCUL5JWYCY6R6WTWYUEUVLA, seg_pos={0 0}, upload=449478]: Long tail cancellation due to optimal threshold Dec 26 23:43:37.524594558 (512h14m37.395677593s) [noise=true , address=145.79.227.2:4115 , piece_id=JFKOTVWUVHFPPYUJX3GGAELRCRWBNRLQRER5TCVEMDG2HADE5GUA, seg_pos={0 0}, upload=449478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: write tcp 188.166.225.170:55956->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55956->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55956->145.79.227.2:4115: use of closed network connection Dec 26 23:43:37.524614055 (512h14m37.39569709s) [noise=true , address=145.79.227.2:4115 , piece_id=JFKOTVWUVHFPPYUJX3GGAELRCRWBNRLQRER5TCVEMDG2HADE5GUA, seg_pos={0 0}, upload=449478]: Long tail cancellation due to optimal threshold Dec 26 23:43:37.524824275 (512h14m37.39590731s) [noise=true , address=45.91.116.14:4189 , piece_id=5Q4HPMF35MDOX5GRDFAMPCFRNAG4VIMT2RADQPGIHLWXBPAWUO3Q, seg_pos={0 0}, upload=449478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: expected piece hash; context canceled; EOF Dec 26 23:43:37.524833503 (512h14m37.395916578s) [noise=true , address=45.91.116.14:4189 , piece_id=5Q4HPMF35MDOX5GRDFAMPCFRNAG4VIMT2RADQPGIHLWXBPAWUO3Q, seg_pos={0 0}, upload=449478]: Long tail cancellation due to optimal threshold Dec 26 23:43:37.524857398 (512h14m37.395940443s) [seg_pos={0 0}, upload=449478]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:43:37.524945546 (512h14m37.396028581s) [upload=449478]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:43:37.524952099 (512h14m37.396035154s) [upload=449478]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:43:37.830467288 (512h14m37.701550333s) [upload=449478]: Done uploading segments... Dec 26 23:43:37.830518857 (512h14m37.701601892s) [upload=449478]: Upload finished. err: <nil> Duration: 30.003916106s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.42906242s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:43:52 UTC) orbiter warning: context deadline exceeded Duration: 6.42908895s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.203991146s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:43:28 UTC) orbiter warning: context deadline exceeded Duration: 6.204017937s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.288280386s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:43:05 UTC) orbiter warning: context deadline exceeded Duration: 6.288364466s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.......................X.X.X.X.XX....X.X.X.......X..XX.XX..X] duration: 30.002787079s trace: https://jaeger.dev.storj.io/trace/6c69d621d8eb8b84 FAIL (2025-12-26 23:42:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 42:26.060643756 (512h13m25.931726811s) [upload=449463]: Starting upload Dec 26 23:42:26.060800073 (512h13m25.931883118s) [upload=449463]: Done starting upload Dec 26 23:42:26.061208772 (512h13m25.932291837s) [upload=449463]: Uploading segments... Dec 26 23:42:26.061269167 (512h13m25.932352212s) [upload=449463]: Got next segment. Inline: false Dec 26 23:42:26.061306107 (512h13m25.932389152s) [upload=449463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:42:26.061311487 (512h13m25.932394542s) [upload=449463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:42:26.318594655 (512h13m26.1896777s) [seg_pos={0 0}, upload=449463]: Begin upload segment... Dec 26 23:42:26.318683124 (512h13m26.189766169s) [seg_pos={0 0}, upload=449463]: Done begin upload segment. Dec 26 23:42:26.318712559 (512h13m26.189795604s) [noise=true , address=145.79.227.23:4545 , piece_id=NHZIKZAXQTY4Z2PT2WZ7HB2C6FK53ZSULQOSFXXTYHGQ624U6ADA, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.318753177 (512h13m26.189836212s) [noise=true , address=145.79.227.16:4910 , piece_id=6MS663WA6NSEO6QNBYOASPKQBNVGLNW27VFAWFZEAHMHUFXLTWNQ, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.318873756 (512h13m26.189956792s) [noise=true , address=145.79.227.47:4265 , piece_id=WMUA7XOIJBUHQFB46WLROZNEL2MEAPLVMD52UNH5S6GRRPMWDLPA, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.319372606 (512h13m26.190455651s) [noise=true , address=145.79.227.50:4374 , piece_id=RPBE5ZAG22MWSTCKV6W7N6IUEV3VAHPXJCRBF4YMBHW7L25U4TBA, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.319551167 (512h13m26.190634202s) [noise=true , address=145.79.227.34:4907 , piece_id=ZD542HENYHQUJAW5CWAIJGEPKLOL3ADQW4FJH6ESREH6LJG5GLMA, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.319899079 (512h13m26.190982114s) [noise=true , address=145.79.227.37:4567 , piece_id=CXYM6VEUC62ASHPONA5MIKHVJ7IDCM7WS7ZJQNUGYZEUJKG457QQ, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.320224609 (512h13m26.191307644s) [noise=true , address=145.79.227.17:4971 , piece_id=HZCFAEC7EV4U47WA7UC23EARSTUNKIQZWC3GXMAGN37PSVJX6JBA, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.320414640 (512h13m26.191497685s) [noise=true , address=145.79.227.5:4222 , piece_id=6ATUR73NEWV545FQRFH7ETVKCKLP2KCW5VM4ZB3ZW63AHEGQEUPQ, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.320526143 (512h13m26.191609178s) [noise=true , address=77.90.31.17:4773 , piece_id=2JFFLEUNXEIET4DBRBVFUZNWGALEDDW2CKNIW3DAP2VSTGWWOTDA, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.320866801 (512h13m26.191949836s) [noise=true , address=145.79.227.46:4602 , piece_id=24XD5MZ7UTGUNPQSGCRNBRMRYJBC3FF2PY2GEX3ECGLSDY2HU55A, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.320980919 (512h13m26.192063964s) [noise=true , address=45.91.116.6:4976 , piece_id=2QN2VYMQSYXT2CL6DCIWNX4K2DXOFL6XRVT5QA3EOAAW6VMMNZEA, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.321141935 (512h13m26.19222498s) [noise=true , address=145.79.227.15:4875 , piece_id=NQA5BDRIIE3RYTCSLMC52TKWEOLVZ2WI727YXEAHMUXU4HCTWZ5Q, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.321395559 (512h13m26.192478604s) [noise=true , address=45.91.116.13:4156 , piece_id=QKGQKZXNU4BDVEJRXT4HK2NEKNYWIP4Z4H3WK3OLKXVAOFOA2U6A, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.321477344 (512h13m26.192560389s) [noise=true , address=77.90.31.16:4736 , piece_id=4CVFBG5CKVA3GLAYWVJGLDMEBCKXZ3LJTUI2T4H3TPW6UYMAH7SA, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.322173400 (512h13m26.193256435s) [noise=true , address=45.91.116.11:4120 , piece_id=XZBEGMPQKMCO7CVZEXXKIWCU4WI4XROO7WGWEMJFJH72S3AGIQBQ, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:26.322806515 (512h13m26.19388956s) [noise=true , address=145.79.227.39:4014 , piece_id=O5DMEW4GDNCXY4H7W2XY7IUSFP3EXRICJCA4SJMGX6EAG47KVH3Q, seg_pos={0 0}, upload=449463]: Uploading piece... Dec 26 23:42:27.946879740 (512h13m27.817962785s) [upload=449463]: Next returned nil segment Dec 26 23:42:27.946894107 (512h13m27.817977142s) [upload=449463]: Waiting for error group managing segments... Dec 26 23:42:28.301744518 (512h13m28.172827553s) [noise=true , address=145.79.227.46:4602 , piece_id=24XD5MZ7UTGUNPQSGCRNBRMRYJBC3FF2PY2GEX3ECGLSDY2HU55A, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:28.308211803 (512h13m28.179294858s) [noise=true , address=145.79.227.47:4265 , piece_id=WMUA7XOIJBUHQFB46WLROZNEL2MEAPLVMD52UNH5S6GRRPMWDLPA, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:28.311337706 (512h13m28.182420741s) [noise=true , address=145.79.227.15:4875 , piece_id=NQA5BDRIIE3RYTCSLMC52TKWEOLVZ2WI727YXEAHMUXU4HCTWZ5Q, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:28.320061166 (512h13m28.191144201s) [noise=true , address=145.79.227.50:4374 , piece_id=RPBE5ZAG22MWSTCKV6W7N6IUEV3VAHPXJCRBF4YMBHW7L25U4TBA, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:28.321085567 (512h13m28.192168612s) [noise=true , address=145.79.227.37:4567 , piece_id=CXYM6VEUC62ASHPONA5MIKHVJ7IDCM7WS7ZJQNUGYZEUJKG457QQ, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:28.322185832 (512h13m28.193268867s) [noise=true , address=145.79.227.16:4910 , piece_id=6MS663WA6NSEO6QNBYOASPKQBNVGLNW27VFAWFZEAHMHUFXLTWNQ, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:28.687155692 (512h13m28.558238737s) [noise=true , address=45.91.116.13:4156 , piece_id=QKGQKZXNU4BDVEJRXT4HK2NEKNYWIP4Z4H3WK3OLKXVAOFOA2U6A, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:28.698948408 (512h13m28.570031453s) [noise=true , address=45.91.116.6:4976 , piece_id=2QN2VYMQSYXT2CL6DCIWNX4K2DXOFL6XRVT5QA3EOAAW6VMMNZEA, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:28.768852947 (512h13m28.639935992s) [noise=true , address=45.91.116.11:4120 , piece_id=XZBEGMPQKMCO7CVZEXXKIWCU4WI4XROO7WGWEMJFJH72S3AGIQBQ, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:29.159406854 (512h13m29.030489899s) [noise=true , address=145.79.227.17:4971 , piece_id=HZCFAEC7EV4U47WA7UC23EARSTUNKIQZWC3GXMAGN37PSVJX6JBA, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:29.196897232 (512h13m29.067980267s) [noise=true , address=77.90.31.17:4773 , piece_id=2JFFLEUNXEIET4DBRBVFUZNWGALEDDW2CKNIW3DAP2VSTGWWOTDA, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:29.281246776 (512h13m29.152329811s) [noise=true , address=77.90.31.16:4736 , piece_id=4CVFBG5CKVA3GLAYWVJGLDMEBCKXZ3LJTUI2T4H3TPW6UYMAH7SA, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:29.701359117 (512h13m29.572442152s) [noise=true , address=145.79.227.39:4014 , piece_id=O5DMEW4GDNCXY4H7W2XY7IUSFP3EXRICJCA4SJMGX6EAG47KVH3Q, seg_pos={0 0}, upload=449463]: Done uploading piece. err: <nil> Dec 26 23:42:29.701384816 (512h13m29.572467861s) [seg_pos={0 0}, upload=449463]: Segment reached optimal threshold of 13 pieces. Dec 26 23:42:29.701737107 (512h13m29.572820142s) [noise=true , address=145.79.227.23:4545 , piece_id=NHZIKZAXQTY4Z2PT2WZ7HB2C6FK53ZSULQOSFXXTYHGQ624U6ADA, seg_pos={0 0}, upload=449463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: write tcp 188.166.225.170:40934->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:40934->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40934->145.79.227.23:4545: use of closed network connection Dec 26 23:42:29.701754089 (512h13m29.572837135s) [noise=true , address=145.79.227.23:4545 , piece_id=NHZIKZAXQTY4Z2PT2WZ7HB2C6FK53ZSULQOSFXXTYHGQ624U6ADA, seg_pos={0 0}, upload=449463]: Long tail cancellation due to optimal threshold Dec 26 23:42:29.701792813 (512h13m29.572875858s) [noise=true , address=145.79.227.5:4222 , piece_id=6ATUR73NEWV545FQRFH7ETVKCKLP2KCW5VM4ZB3ZW63AHEGQEUPQ, seg_pos={0 0}, upload=449463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1): protocol: expected piece hash; context canceled; EOF Dec 26 23:42:29.701898164 (512h13m29.572981229s) [noise=true , address=145.79.227.5:4222 , piece_id=6ATUR73NEWV545FQRFH7ETVKCKLP2KCW5VM4ZB3ZW63AHEGQEUPQ, seg_pos={0 0}, upload=449463]: Long tail cancellation due to optimal threshold Dec 26 23:42:29.702180972 (512h13m29.573264017s) [noise=true , address=145.79.227.34:4907 , piece_id=ZD542HENYHQUJAW5CWAIJGEPKLOL3ADQW4FJH6ESREH6LJG5GLMA, seg_pos={0 0}, upload=449463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF Dec 26 23:42:29.702190029 (512h13m29.573273074s) [noise=true , address=145.79.227.34:4907 , piece_id=ZD542HENYHQUJAW5CWAIJGEPKLOL3ADQW4FJH6ESREH6LJG5GLMA, seg_pos={0 0}, upload=449463]: Long tail cancellation due to optimal threshold Dec 26 23:42:29.702212632 (512h13m29.573295668s) [seg_pos={0 0}, upload=449463]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:42:29.702353811 (512h13m29.573436856s) [upload=449463]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:42:29.702391172 (512h13m29.573474217s) [upload=449463]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:42:30.006010772 (512h13m29.877093817s) [upload=449463]: Done uploading segments... Dec 26 23:42:30.006031542 (512h13m29.877114587s) [upload=449463]: Upload finished. err: <nil> Duration: 30.00281385s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.274676274s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:42:28 UTC) orbiter warning: context deadline exceeded Duration: 6.274704046s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.071658011s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:41:52 UTC) orbiter warning: context deadline exceeded Duration: 6.071686506s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [........................X.X.X.X.XX....X.X.X.......X..XX.XX..] duration: 8.279459609s trace: https://jaeger.dev.storj.io/trace/5f324224e2d6014c FAIL (2025-12-26 23:41:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: ZNCWKORHC2HET6UEDU4BYZ5GANCGWW4ISYX73KJT4KEK5HE22A5Q) context deadline exceeded; piecestore: (Node ID: 12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, Piece ID: PNXVWKO7MIRQJSACOIKD4KYJ2YISYEOZAJCWOQNNUUNR32QLDXHQ) context deadline exceeded ======LOGS====== c 26 23:40:10.181697335 (512h11m10.05278038s) [upload=449432]: Starting upload Dec 26 23:40:10.181776015 (512h11m10.05285907s) [upload=449432]: Done starting upload Dec 26 23:40:10.182060557 (512h11m10.053143592s) [upload=449432]: Uploading segments... Dec 26 23:40:10.182168462 (512h11m10.053251507s) [upload=449432]: Got next segment. Inline: false Dec 26 23:40:10.182217585 (512h11m10.05330063s) [upload=449432]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 26 23:40:10.182227203 (512h11m10.053310248s) [upload=449432]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 26 23:40:10.428448126 (512h11m10.299531161s) [seg_pos={0 0}, upload=449432]: Begin upload segment... Dec 26 23:40:10.428501307 (512h11m10.299584342s) [seg_pos={0 0}, upload=449432]: Done begin upload segment. Dec 26 23:40:10.428528328 (512h11m10.299611373s) [noise=true , address=145.79.227.15:4874 , piece_id=HPWQLXPWMO2WSH7CZJYPMKNWNVDP2LDEUKTDRILLE64PUISWXN3Q, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.429463828 (512h11m10.300546874s) [noise=true , address=145.79.227.48:4301 , piece_id=JX2CAXGSKSNHTNJLQ5NT3K6T2VGIUUD7LVQWRCQ6XDROFY4MRZPA, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.430121902 (512h11m10.301204937s) [noise=true , address=145.79.227.42:4123 , piece_id=EBGMWDY4VLV3YW6LY5Q6MAFN3UGHINOQDX5T2WF2AH22OWRGE4GQ, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.430713708 (512h11m10.301796743s) [noise=true , address=45.91.116.8:4048 , piece_id=C76ST5AIA7L32VZPVBUFSCAEPL4JATDR2T6MJN7KSGBD2TRCPE4A, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.431164977 (512h11m10.302248022s) [noise=true , address=145.79.227.6:4259 , piece_id=N7EYHLTF4QS4WIUASDVNDAJ27DQ4FAXS7CD5Q5URCHBPLLQ6G5CQ, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.431559628 (512h11m10.302642673s) [noise=true , address=77.90.31.21:4880 , piece_id=XCF37ZFMXOFBTRUQCTLS6GVPTUB54XAF4NWOEL65GR45DXED5MGA, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.432115485 (512h11m10.303198531s) [noise=true , address=45.91.116.14:4190 , piece_id=ZNCWKORHC2HET6UEDU4BYZ5GANCGWW4ISYX73KJT4KEK5HE22A5Q, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.432568198 (512h11m10.303651243s) [noise=true , address=145.79.227.26:4653 , piece_id=C7HW5PVDRGPVUP3J63U2W4U4D6YK5PFIE22KABDYCC5CW6B2LEAA, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.433139715 (512h11m10.30422275s) [noise=true , address=77.90.31.17:4773 , piece_id=U6KJQZVM2A4BHGYNY5C5DBT46B5Z3MA5KMAR3Z6U7BJ2ZSNRMUKQ, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.433570415 (512h11m10.30465346s) [noise=true , address=45.91.116.13:4156 , piece_id=JQC7EXSVNBYBR6UTBK6SAICBG54W6OFOOKUVHGAOU57NMWJPBSPA, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.434730022 (512h11m10.305813097s) [noise=true , address=45.91.116.5:4298 , piece_id=PNXVWKO7MIRQJSACOIKD4KYJ2YISYEOZAJCWOQNNUUNR32QLDXHQ, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.435362345 (512h11m10.30644538s) [noise=true , address=145.79.227.13:4802 , piece_id=2Q74ZAYORPVLUQJWNXUIW566VTYTM47J63VUUQLD2CVGO35T3DFQ, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.435884529 (512h11m10.306967574s) [noise=true , address=145.79.227.11:4732 , piece_id=2MBP7FMZV73JA2VLSL6BIWVDQSZRNUN6KU2VHVPPKNUA7ANW42OQ, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.436499850 (512h11m10.307582885s) [noise=true , address=145.79.227.36:4517 , piece_id=PYQOQEOC6NRXGV6EUYR2HCGHCJWX2SCIQYMEZ3WCY2F3FH5FQWKQ, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.436962411 (512h11m10.308045456s) [noise=true , address=145.79.227.23:4545 , piece_id=5SBVBGUWZDCYO65HNYZT75SZRQCLGBRXNYTCIAITBTQQ6H3VCTVA, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:10.437344988 (512h11m10.308428033s) [noise=true , address=145.79.227.47:4265 , piece_id=AJX5RPQGPJP7NSI26HKC7PWXNNLU5TTCTMV3UH563HLLNPII6VMQ, seg_pos={0 0}, upload=449432]: Uploading piece... Dec 26 23:40:12.047140092 (512h11m11.918223147s) [upload=449432]: Next returned nil segment Dec 26 23:40:12.047155672 (512h11m11.918238707s) [upload=449432]: Waiting for error group managing segments... Dec 26 23:40:12.428247533 (512h11m12.299330568s) [noise=true , address=145.79.227.11:4732 , piece_id=2MBP7FMZV73JA2VLSL6BIWVDQSZRNUN6KU2VHVPPKNUA7ANW42OQ, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.434863955 (512h11m12.30594699s) [noise=true , address=145.79.227.42:4123 , piece_id=EBGMWDY4VLV3YW6LY5Q6MAFN3UGHINOQDX5T2WF2AH22OWRGE4GQ, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.436102382 (512h11m12.307185427s) [noise=true , address=145.79.227.13:4802 , piece_id=2Q74ZAYORPVLUQJWNXUIW566VTYTM47J63VUUQLD2CVGO35T3DFQ, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.446441029 (512h11m12.317524064s) [noise=true , address=145.79.227.26:4653 , piece_id=C7HW5PVDRGPVUP3J63U2W4U4D6YK5PFIE22KABDYCC5CW6B2LEAA, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.457596440 (512h11m12.328679485s) [noise=true , address=145.79.227.6:4259 , piece_id=N7EYHLTF4QS4WIUASDVNDAJ27DQ4FAXS7CD5Q5URCHBPLLQ6G5CQ, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.458538112 (512h11m12.329621157s) [noise=true , address=145.79.227.15:4874 , piece_id=HPWQLXPWMO2WSH7CZJYPMKNWNVDP2LDEUKTDRILLE64PUISWXN3Q, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.459937856 (512h11m12.331020891s) [noise=true , address=145.79.227.47:4265 , piece_id=AJX5RPQGPJP7NSI26HKC7PWXNNLU5TTCTMV3UH563HLLNPII6VMQ, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.460266141 (512h11m12.331349186s) [noise=true , address=145.79.227.48:4301 , piece_id=JX2CAXGSKSNHTNJLQ5NT3K6T2VGIUUD7LVQWRCQ6XDROFY4MRZPA, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.464011349 (512h11m12.335094394s) [noise=true , address=145.79.227.36:4517 , piece_id=PYQOQEOC6NRXGV6EUYR2HCGHCJWX2SCIQYMEZ3WCY2F3FH5FQWKQ, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.681530916 (512h11m12.552613951s) [noise=true , address=45.91.116.14:4190 , piece_id=ZNCWKORHC2HET6UEDU4BYZ5GANCGWW4ISYX73KJT4KEK5HE22A5Q, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.700113784 (512h11m12.571196829s) [noise=true , address=45.91.116.5:4298 , piece_id=PNXVWKO7MIRQJSACOIKD4KYJ2YISYEOZAJCWOQNNUUNR32QLDXHQ, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.700613886 (512h11m12.571696931s) [noise=true , address=45.91.116.8:4048 , piece_id=C76ST5AIA7L32VZPVBUFSCAEPL4JATDR2T6MJN7KSGBD2TRCPE4A, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.771668328 (512h11m12.642751363s) [noise=true , address=45.91.116.13:4156 , piece_id=JQC7EXSVNBYBR6UTBK6SAICBG54W6OFOOKUVHGAOU57NMWJPBSPA, seg_pos={0 0}, upload=449432]: Done uploading piece. err: <nil> Dec 26 23:40:12.771692223 (512h11m12.642775258s) [seg_pos={0 0}, upload=449432]: Segment reached optimal threshold of 13 pieces. Dec 26 23:40:12.771891273 (512h11m12.642974308s) [noise=true , address=77.90.31.17:4773 , piece_id=U6KJQZVM2A4BHGYNY5C5DBT46B5Z3MA5KMAR3Z6U7BJ2ZSNRMUKQ, seg_pos={0 0}, upload=449432]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ): protocol: expected piece hash; context canceled; EOF Dec 26 23:40:12.771898015 (512h11m12.64298105s) [noise=true , address=77.90.31.17:4773 , piece_id=U6KJQZVM2A4BHGYNY5C5DBT46B5Z3MA5KMAR3Z6U7BJ2ZSNRMUKQ, seg_pos={0 0}, upload=449432]: Long tail cancellation due to optimal threshold Dec 26 23:40:12.772002725 (512h11m12.64308577s) [noise=true , address=77.90.31.21:4880 , piece_id=XCF37ZFMXOFBTRUQCTLS6GVPTUB54XAF4NWOEL65GR45DXED5MGA, seg_pos={0 0}, upload=449432]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb): protocol: expected piece hash; context canceled; EOF Dec 26 23:40:12.772014066 (512h11m12.643097111s) [noise=true , address=77.90.31.21:4880 , piece_id=XCF37ZFMXOFBTRUQCTLS6GVPTUB54XAF4NWOEL65GR45DXED5MGA, seg_pos={0 0}, upload=449432]: Long tail cancellation due to optimal threshold Dec 26 23:40:12.772122302 (512h11m12.643205347s) [noise=true , address=145.79.227.23:4545 , piece_id=5SBVBGUWZDCYO65HNYZT75SZRQCLGBRXNYTCIAITBTQQ6H3VCTVA, seg_pos={0 0}, upload=449432]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: write tcp 188.166.225.170:55482->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:55482->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55482->145.79.227.23:4545: use of closed network connection Dec 26 23:40:12.772132271 (512h11m12.643215317s) [noise=true , address=145.79.227.23:4545 , piece_id=5SBVBGUWZDCYO65HNYZT75SZRQCLGBRXNYTCIAITBTQQ6H3VCTVA, seg_pos={0 0}, upload=449432]: Long tail cancellation due to optimal threshold Dec 26 23:40:12.772155455 (512h11m12.64323849s) [seg_pos={0 0}, upload=449432]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 26 23:40:12.772238944 (512h11m12.643321989s) [upload=449432]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 26 23:40:12.772245327 (512h11m12.643328372s) [upload=449432]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 26 23:40:13.070473189 (512h11m12.941556224s) [upload=449432]: Done uploading segments... Dec 26 23:40:13.070496414 (512h11m12.941579459s) [upload=449432]: Upload finished. err: <nil> Duration: 8.279477344s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.362548795s trace: https://jaeger.dev.storj.io/trace/f21975653b9b600 FAIL (2025-12-26 23:41:31 UTC) orbiter warning: context deadline exceeded Duration: 6.362576137s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)