Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X..X........X......X.X..............X...........XXXX.] duration: 24.23431921s trace: https://jaeger.dev.storj.io/trace/3f721ec657861326 FAIL (2026-01-08 12:58:36 UTC) orbiter warning: context deadline exceeded Duration: 24.234495016s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [..............X.......X.............................X...XX..] duration: 28.215436191s trace: https://jaeger.dev.storj.io/trace/286f626e9aabfcfb FAIL (2026-01-08 12:58:32 UTC) orbiter warning: context deadline exceeded Duration: 28.215465365s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X................X.........X..........X.X........XXX......XX] duration: 30.002633737s trace: https://jaeger.dev.storj.io/trace/6b35eacf607464b2 FAIL (2026-01-08 12:58:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, Piece ID: 37HHRLXV7ULP4GXSYS7OJOO6AXJ6OUUPTLCCEPHK44OIH4IIIMEQ) context deadline exceeded; piecestore: (Node ID: 1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, Piece ID: EJW4HVKMZJXGLRIZVVXUUJYTYWVVAAXWZMQCBGMUBQHEXOVR23BA) context deadline exceeded ======LOGS====== Jan 8 12:57:57.318185292 (813h28m57.189268337s) [upload=711548]: Done starting upload Jan 8 12:57:57.318225057 (813h28m57.189308092s) [upload=711548]: Uploading segments... Jan 8 12:57:57.318420860 (813h28m57.189503906s) [upload=711548]: Got next segment. Inline: false Jan 8 12:57:57.318455647 (813h28m57.189538692s) [upload=711548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:57:57.318463562 (813h28m57.189546607s) [upload=711548]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:57:57.565096172 (813h28m57.436179207s) [seg_pos={0 0}, upload=711548]: Begin upload segment... Jan 8 12:57:57.565150365 (813h28m57.4362334s) [seg_pos={0 0}, upload=711548]: Done begin upload segment. Jan 8 12:57:57.565190241 (813h28m57.436273276s) [noise=true , address=145.79.227.40:4050 , piece_id=XGWD35T4E6QJY7LW7BKPKHSRL7JH5GJQLITNE5XBEV4KQNSLKD2A, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.565266015 (813h28m57.43634906s) [noise=true , address=145.79.227.3:4150 , piece_id=XWO4ELQPXUZAMJ6CKFGHPSUMDKI2LFXPHPMCLTKG5IMUHQBXEPSA, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.566222716 (813h28m57.437305762s) [noise=true , address=145.79.227.5:4223 , piece_id=LY2LB6CF6IEZO6AQLT37WVOPCEWQHOEA72KLVYXZ6HVCHWC67M4A, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.566303251 (813h28m57.437386286s) [noise=true , address=77.90.31.21:4880 , piece_id=PSOCN3OIIA5CR5DMSNS2I3NOP7VDXJZF2DWBD4T3UGPM74VIUYZQ, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.566734451 (813h28m57.437817506s) [noise=true , address=145.79.227.13:4803 , piece_id=ASLARSPACCHY4OFNHYI3LWOGIAHQTHGVRXUHNMBAGH3EARQZEZYQ, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.567118322 (813h28m57.438201357s) [noise=true , address=145.79.227.14:4837 , piece_id=BVLJVQA4TPLZSNY3L6MO2KOBKHZN3T7NXG3FTME5KNYTB4YTCVDQ, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.567524576 (813h28m57.438607611s) [noise=true , address=77.90.31.16:4735 , piece_id=37HHRLXV7ULP4GXSYS7OJOO6AXJ6OUUPTLCCEPHK44OIH4IIIMEQ, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.567914178 (813h28m57.438997213s) [noise=true , address=145.79.227.49:4337 , piece_id=XKUFSYWIHHDITQXL6O7M3HQ7L6J2NUP5VH5QLSQP4NW3ZOATG5XA, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.568318337 (813h28m57.439401372s) [noise=true , address=145.79.227.35:4483 , piece_id=5YE4WPFXUEEE2AUKNPKTB3P2WSCGXEINXJSS6TVCJMO36INDKYIA, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.568749759 (813h28m57.439832794s) [noise=true , address=145.79.227.12:4767 , piece_id=ANVYWYUNJZB35CCNRUHK3BKRFQ3TUKKSTP3CH2WCGUS7IQLMDOIQ, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.570269752 (813h28m57.441352788s) [noise=true , address=45.91.116.9:4263 , piece_id=6ZAB5X3JOKVRTY3A6LQORDWO22OHDFAOUWMC3UJ5ZLV5KJP2MVVQ, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.570594310 (813h28m57.441677345s) [noise=true , address=145.79.227.10:4403 , piece_id=EA36OOSSRTMFNKDD5SFWIMC5EKYTC7KAEGOSRKSPK4T6AWDF465Q, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.570922235 (813h28m57.44200527s) [noise=true , address=77.90.31.10:4520 , piece_id=EJW4HVKMZJXGLRIZVVXUUJYTYWVVAAXWZMQCBGMUBQHEXOVR23BA, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.571199593 (813h28m57.442282628s) [noise=true , address=45.91.116.11:4117 , piece_id=4K3QOOSZ4PBEM246YPYIXOH6WWACCND6H7PLB5A2SPUCQOUEQV6A, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.571469908 (813h28m57.442552943s) [noise=true , address=145.79.227.44:4193 , piece_id=TS4LKXUMJDDJXXK34BAU3JDNRZIII46URQPVLHJ3OYO4UNTMLYLA, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:57.571755873 (813h28m57.442838908s) [noise=true , address=145.79.227.21:4473 , piece_id=QHTVFZEAZ4TYXDKQU7CQYMD6MFM3PZ3H2QHCIBYRTNN7JDT4XZQA, seg_pos={0 0}, upload=711548]: Uploading piece... Jan 8 12:57:59.248279201 (813h28m59.119362247s) [upload=711548]: Next returned nil segment Jan 8 12:57:59.248302907 (813h28m59.119385972s) [upload=711548]: Waiting for error group managing segments... Jan 8 12:57:59.544457434 (813h28m59.415540469s) [noise=true , address=145.79.227.35:4483 , piece_id=5YE4WPFXUEEE2AUKNPKTB3P2WSCGXEINXJSS6TVCJMO36INDKYIA, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.548912016 (813h28m59.419995051s) [noise=true , address=145.79.227.49:4337 , piece_id=XKUFSYWIHHDITQXL6O7M3HQ7L6J2NUP5VH5QLSQP4NW3ZOATG5XA, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.552309284 (813h28m59.42339232s) [noise=true , address=145.79.227.14:4837 , piece_id=BVLJVQA4TPLZSNY3L6MO2KOBKHZN3T7NXG3FTME5KNYTB4YTCVDQ, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.575862483 (813h28m59.446945518s) [noise=true , address=145.79.227.40:4050 , piece_id=XGWD35T4E6QJY7LW7BKPKHSRL7JH5GJQLITNE5XBEV4KQNSLKD2A, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.578492411 (813h28m59.449575446s) [noise=true , address=145.79.227.3:4150 , piece_id=XWO4ELQPXUZAMJ6CKFGHPSUMDKI2LFXPHPMCLTKG5IMUHQBXEPSA, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.581525656 (813h28m59.452608701s) [noise=true , address=145.79.227.12:4767 , piece_id=ANVYWYUNJZB35CCNRUHK3BKRFQ3TUKKSTP3CH2WCGUS7IQLMDOIQ, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.586330966 (813h28m59.457414001s) [noise=true , address=145.79.227.21:4473 , piece_id=QHTVFZEAZ4TYXDKQU7CQYMD6MFM3PZ3H2QHCIBYRTNN7JDT4XZQA, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.637902486 (813h28m59.508985521s) [noise=true , address=145.79.227.5:4223 , piece_id=LY2LB6CF6IEZO6AQLT37WVOPCEWQHOEA72KLVYXZ6HVCHWC67M4A, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.639602362 (813h28m59.510685407s) [noise=true , address=145.79.227.10:4403 , piece_id=EA36OOSSRTMFNKDD5SFWIMC5EKYTC7KAEGOSRKSPK4T6AWDF465Q, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.649469981 (813h28m59.520553016s) [noise=true , address=145.79.227.13:4803 , piece_id=ASLARSPACCHY4OFNHYI3LWOGIAHQTHGVRXUHNMBAGH3EARQZEZYQ, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.853352360 (813h28m59.724435395s) [noise=true , address=77.90.31.16:4735 , piece_id=37HHRLXV7ULP4GXSYS7OJOO6AXJ6OUUPTLCCEPHK44OIH4IIIMEQ, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.861173352 (813h28m59.732256397s) [noise=true , address=77.90.31.21:4880 , piece_id=PSOCN3OIIA5CR5DMSNS2I3NOP7VDXJZF2DWBD4T3UGPM74VIUYZQ, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.867610278 (813h28m59.738693324s) [noise=true , address=77.90.31.10:4520 , piece_id=EJW4HVKMZJXGLRIZVVXUUJYTYWVVAAXWZMQCBGMUBQHEXOVR23BA, seg_pos={0 0}, upload=711548]: Done uploading piece. err: <nil> Jan 8 12:57:59.867623414 (813h28m59.738706449s) [seg_pos={0 0}, upload=711548]: Segment reached optimal threshold of 13 pieces. Jan 8 12:57:59.867812774 (813h28m59.738895819s) [noise=true , address=145.79.227.44:4193 , piece_id=TS4LKXUMJDDJXXK34BAU3JDNRZIII46URQPVLHJ3OYO4UNTMLYLA, seg_pos={0 0}, upload=711548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: write tcp 188.166.225.170:33588->145.79.227.44:4193: use of closed network connection; write tcp 188.166.225.170:33588->145.79.227.44:4193: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33588->145.79.227.44:4193: use of closed network connection Jan 8 12:57:59.867819998 (813h28m59.738903043s) [noise=true , address=145.79.227.44:4193 , piece_id=TS4LKXUMJDDJXXK34BAU3JDNRZIII46URQPVLHJ3OYO4UNTMLYLA, seg_pos={0 0}, upload=711548]: Long tail cancellation due to optimal threshold Jan 8 12:57:59.867936360 (813h28m59.739019395s) [noise=true , address=45.91.116.9:4263 , piece_id=6ZAB5X3JOKVRTY3A6LQORDWO22OHDFAOUWMC3UJ5ZLV5KJP2MVVQ, seg_pos={0 0}, upload=711548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Jan 8 12:57:59.867951939 (813h28m59.739034984s) [noise=true , address=45.91.116.9:4263 , piece_id=6ZAB5X3JOKVRTY3A6LQORDWO22OHDFAOUWMC3UJ5ZLV5KJP2MVVQ, seg_pos={0 0}, upload=711548]: Long tail cancellation due to optimal threshold Jan 8 12:57:59.868133314 (813h28m59.739216359s) [noise=true , address=45.91.116.11:4117 , piece_id=4K3QOOSZ4PBEM246YPYIXOH6WWACCND6H7PLB5A2SPUCQOUEQV6A, seg_pos={0 0}, upload=711548]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: rpc: dial tcp 45.91.116.11:4117: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jan 8 12:57:59.868141931 (813h28m59.739224976s) [noise=true , address=45.91.116.11:4117 , piece_id=4K3QOOSZ4PBEM246YPYIXOH6WWACCND6H7PLB5A2SPUCQOUEQV6A, seg_pos={0 0}, upload=711548]: Long tail cancellation due to optimal threshold Jan 8 12:57:59.868171317 (813h28m59.739254352s) [seg_pos={0 0}, upload=711548]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:57:59.868243183 (813h28m59.739326218s) [upload=711548]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:57:59.868249146 (813h28m59.739332181s) [upload=711548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:58:00.164430272 (813h29m0.035513317s) [upload=711548]: Done uploading segments... Jan 8 12:58:00.164450431 (813h29m0.035533466s) [upload=711548]: Upload finished. err: <nil> Duration: 30.002655678s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........X..X........X......X.X..............X...........XXXX] duration: 30.001798315s trace: https://jaeger.dev.storj.io/trace/3f721ec657861326 FAIL (2026-01-08 12:58:06 UTC) orbiter warning: context deadline exceeded Duration: 30.001818733s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [X.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXXXXXXX.XX] duration: 5.001481807s trace: https://jaeger.dev.storj.io/trace/7021aadacde1177 FAIL (2026-01-08 12:57:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001501596s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........X..X........X......X.X..............X...........XXX] duration: 30.001499192s trace: https://jaeger.dev.storj.io/trace/3c56a90635d3de6 FAIL (2026-01-08 12:57:30 UTC) orbiter warning: context deadline exceeded Duration: 30.001528006s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [...............X.......X.............................X...XX.] duration: 7.747392971s trace: https://jaeger.dev.storj.io/trace/286f626e9aabfcfb FAIL (2026-01-08 12:57:29 UTC) orbiter warning: context deadline exceeded Duration: 7.747406345s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XX................X.........X..........X.X........XXX......X] duration: 30.002589105s trace: https://jaeger.dev.storj.io/trace/7d146d04087ef843 FAIL (2026-01-08 12:57:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: Z7DBSSQPTBQB2CSJEMXIEUNP2UJOWHZM2GRKP44NHLVCP65ZQV4Q) context deadline exceeded; piecestore: (Node ID: 1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, Piece ID: KCOFVPOFBMVQ6EBUPLF5TDWDXN2GNRSNYM527P2V3N7SJD2TND4A) context deadline exceeded ======LOGS====== 8 12:56:57.531024151 (813h27m57.402107196s) [upload=711532]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:56:57.779754526 (813h27m57.650837561s) [seg_pos={0 0}, upload=711532]: Begin upload segment... Jan 8 12:56:57.779795905 (813h27m57.65087895s) [seg_pos={0 0}, upload=711532]: Done begin upload segment. Jan 8 12:56:57.779817917 (813h27m57.650900952s) [noise=true , address=145.79.227.15:4876 , piece_id=AIVWOYPNNA2WDGLM3T7AUG5JB4CAZY3IYFC7LWPOY2PIJN5YSQCQ, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.779845079 (813h27m57.650928114s) [noise=true , address=145.79.227.35:4482 , piece_id=3CDLYL26ZD32RVT4LZZ2FMEPXP7KLBX4HRS3A62XJU6H7KZCI3SQ, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.780404324 (813h27m57.651487359s) [noise=true , address=145.79.227.32:4834 , piece_id=PJJ6E7S3NJB5NTXMBCQ627A4FX2LHW4UN35N23LT63ST2P5PFD5Q, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.780650793 (813h27m57.651733828s) [noise=true , address=45.91.116.9:4262 , piece_id=EBYWX2LRE5K2EYD76NNCA7EFVFRGM3B7UPPA7OMDI34E2X4NQQUA, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.781057967 (813h27m57.652141003s) [noise=true , address=145.79.227.39:4014 , piece_id=WMX2CCRAHJOCAYEFER5STFMJFUH6YU43L5VHMCAOGIPNKUSULOSQ, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.781475322 (813h27m57.652558367s) [noise=true , address=145.79.227.10:4403 , piece_id=IJVIAL6PJ64TVWLSKF3WYTVTPIP3EK2YGBX4TNE3M6GTWNOJHORA, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.781588528 (813h27m57.652671573s) [noise=true , address=145.79.227.19:4078 , piece_id=ROLYMD2TWGIWW3UD3PDXZTI5ZKP2KQSEAY6JIHEP4BHDKWLWTSLA, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.782379865 (813h27m57.6534629s) [noise=true , address=145.79.227.11:4729 , piece_id=6M6EFVQ7JY6VEYO63XNK2FJBIX3MAOJKHK3Q734WBXWT36TMYCAA, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.782482129 (813h27m57.653565164s) [noise=true , address=145.79.227.21:4473 , piece_id=MG7DY4SM24U3ZBUXBBYYRFFPP2LLN5AHDXMGIP6T36MP3VGH63TA, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.783186811 (813h27m57.654269846s) [noise=true , address=45.91.116.14:4189 , piece_id=Z7DBSSQPTBQB2CSJEMXIEUNP2UJOWHZM2GRKP44NHLVCP65ZQV4Q, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.783525937 (813h27m57.654608972s) [noise=true , address=77.90.31.10:4520 , piece_id=KSANFBF5DXXGYI6IJ6LNLVZ7LZWZNASMIKCUNLWDKEFDMTYAZNHQ, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.783882786 (813h27m57.654965821s) [noise=true , address=145.79.227.23:4546 , piece_id=4JS7GSUNW5W7QGWWZLHLXMFIFXF2PVEIG7UMZ57ZJMGDSBNOD7QQ, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.784202316 (813h27m57.655285351s) [noise=true , address=45.91.116.12:4228 , piece_id=PLNYWLNOLZHTH64RKP4FY3MSCUTN3V555W23FHF52N2C6CBQ2M6A, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.785499225 (813h27m57.65658226s) [noise=true , address=145.79.227.40:4050 , piece_id=2LLEWQUYSSVVJZLQP36R6ZZ3W3W2NJFAKKRS6PVLMVDH2ZT6QGBQ, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.785869399 (813h27m57.656952434s) [noise=true , address=145.79.227.2:4114 , piece_id=LN7AND6ALYZCDENN5UKN3HJFHE63FTTY6AOLGVU6IOAQAS6OBPSQ, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:57.786300020 (813h27m57.657383055s) [noise=true , address=77.90.31.11:4555 , piece_id=KCOFVPOFBMVQ6EBUPLF5TDWDXN2GNRSNYM527P2V3N7SJD2TND4A, seg_pos={0 0}, upload=711532]: Uploading piece... Jan 8 12:56:59.489073409 (813h27m59.360156454s) [upload=711532]: Next returned nil segment Jan 8 12:56:59.489091484 (813h27m59.360174519s) [upload=711532]: Waiting for error group managing segments... Jan 8 12:56:59.767912185 (813h27m59.63899523s) [noise=true , address=145.79.227.19:4078 , piece_id=ROLYMD2TWGIWW3UD3PDXZTI5ZKP2KQSEAY6JIHEP4BHDKWLWTSLA, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.769400909 (813h27m59.640483944s) [noise=true , address=145.79.227.21:4473 , piece_id=MG7DY4SM24U3ZBUXBBYYRFFPP2LLN5AHDXMGIP6T36MP3VGH63TA, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.771784709 (813h27m59.642867754s) [noise=true , address=145.79.227.15:4876 , piece_id=AIVWOYPNNA2WDGLM3T7AUG5JB4CAZY3IYFC7LWPOY2PIJN5YSQCQ, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.772277637 (813h27m59.643360672s) [noise=true , address=145.79.227.11:4729 , piece_id=6M6EFVQ7JY6VEYO63XNK2FJBIX3MAOJKHK3Q734WBXWT36TMYCAA, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.784134003 (813h27m59.655217048s) [noise=true , address=145.79.227.2:4114 , piece_id=LN7AND6ALYZCDENN5UKN3HJFHE63FTTY6AOLGVU6IOAQAS6OBPSQ, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.784243782 (813h27m59.655326817s) [noise=true , address=145.79.227.32:4834 , piece_id=PJJ6E7S3NJB5NTXMBCQ627A4FX2LHW4UN35N23LT63ST2P5PFD5Q, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.785320583 (813h27m59.656403628s) [noise=true , address=145.79.227.39:4014 , piece_id=WMX2CCRAHJOCAYEFER5STFMJFUH6YU43L5VHMCAOGIPNKUSULOSQ, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.785840753 (813h27m59.656923788s) [noise=true , address=145.79.227.40:4050 , piece_id=2LLEWQUYSSVVJZLQP36R6ZZ3W3W2NJFAKKRS6PVLMVDH2ZT6QGBQ, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.793232999 (813h27m59.664316044s) [noise=true , address=145.79.227.23:4546 , piece_id=4JS7GSUNW5W7QGWWZLHLXMFIFXF2PVEIG7UMZ57ZJMGDSBNOD7QQ, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.852528656 (813h27m59.723611691s) [noise=true , address=145.79.227.35:4482 , piece_id=3CDLYL26ZD32RVT4LZZ2FMEPXP7KLBX4HRS3A62XJU6H7KZCI3SQ, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:56:59.856093103 (813h27m59.727176138s) [noise=true , address=145.79.227.10:4403 , piece_id=IJVIAL6PJ64TVWLSKF3WYTVTPIP3EK2YGBX4TNE3M6GTWNOJHORA, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:57:00.088940334 (813h27m59.960023369s) [noise=true , address=77.90.31.11:4555 , piece_id=KCOFVPOFBMVQ6EBUPLF5TDWDXN2GNRSNYM527P2V3N7SJD2TND4A, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:57:00.342838929 (813h28m0.213921974s) [noise=true , address=45.91.116.14:4189 , piece_id=Z7DBSSQPTBQB2CSJEMXIEUNP2UJOWHZM2GRKP44NHLVCP65ZQV4Q, seg_pos={0 0}, upload=711532]: Done uploading piece. err: <nil> Jan 8 12:57:00.342859068 (813h28m0.213942103s) [seg_pos={0 0}, upload=711532]: Segment reached optimal threshold of 13 pieces. Jan 8 12:57:00.343079608 (813h28m0.214162673s) [noise=true , address=45.91.116.12:4228 , piece_id=PLNYWLNOLZHTH64RKP4FY3MSCUTN3V555W23FHF52N2C6CBQ2M6A, seg_pos={0 0}, upload=711532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: write tcp 188.166.225.170:38934->45.91.116.12:4228: use of closed network connection; write tcp 188.166.225.170:38934->45.91.116.12:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38934->45.91.116.12:4228: use of closed network connection Jan 8 12:57:00.343087764 (813h28m0.214170809s) [noise=true , address=45.91.116.12:4228 , piece_id=PLNYWLNOLZHTH64RKP4FY3MSCUTN3V555W23FHF52N2C6CBQ2M6A, seg_pos={0 0}, upload=711532]: Long tail cancellation due to optimal threshold Jan 8 12:57:00.343176041 (813h28m0.214259087s) [noise=true , address=45.91.116.9:4262 , piece_id=EBYWX2LRE5K2EYD76NNCA7EFVFRGM3B7UPPA7OMDI34E2X4NQQUA, seg_pos={0 0}, upload=711532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: write tcp 188.166.225.170:55976->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:55976->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55976->45.91.116.9:4262: use of closed network connection Jan 8 12:57:00.343192333 (813h28m0.214275368s) [noise=true , address=45.91.116.9:4262 , piece_id=EBYWX2LRE5K2EYD76NNCA7EFVFRGM3B7UPPA7OMDI34E2X4NQQUA, seg_pos={0 0}, upload=711532]: Long tail cancellation due to optimal threshold Jan 8 12:57:00.343208984 (813h28m0.214292019s) [noise=true , address=77.90.31.10:4520 , piece_id=KSANFBF5DXXGYI6IJ6LNLVZ7LZWZNASMIKCUNLWDKEFDMTYAZNHQ, seg_pos={0 0}, upload=711532]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:58794->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:58794->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58794->77.90.31.10:4520: use of closed network connection Jan 8 12:57:00.343219734 (813h28m0.214302789s) [noise=true , address=77.90.31.10:4520 , piece_id=KSANFBF5DXXGYI6IJ6LNLVZ7LZWZNASMIKCUNLWDKEFDMTYAZNHQ, seg_pos={0 0}, upload=711532]: Long tail cancellation due to optimal threshold Jan 8 12:57:00.343312041 (813h28m0.214395076s) [seg_pos={0 0}, upload=711532]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:57:00.343388366 (813h28m0.214471401s) [upload=711532]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:57:00.343393896 (813h28m0.214476931s) [upload=711532]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:57:00.632207585 (813h28m0.503290631s) [upload=711532]: Done uploading segments... Jan 8 12:57:00.632258563 (813h28m0.503341608s) [upload=711532]: Upload finished. err: <nil> Duration:* 30.00261332s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [................X.......X.............................X...XX] duration: 1m0.001702097s trace: https://jaeger.dev.storj.io/trace/286f626e9aabfcfb FAIL (2026-01-08 12:57:21 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001715723s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXXXXXXX.X] duration: 5.00128811s trace: https://jaeger.dev.storj.io/trace/49a79b875e206f8c FAIL (2026-01-08 12:56:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.0013144s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........X..X........X......X.X..............X...........XX] duration: 30.001613013s trace: https://jaeger.dev.storj.io/trace/5743c29b793cbedf FAIL (2026-01-08 12:56:23 UTC) orbiter warning: context deadline exceeded Duration: 30.001642448s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.................X.......X.............................X...X] duration: 1m0.001196167s trace: https://jaeger.dev.storj.io/trace/748d8e33aaab2bb FAIL (2026-01-08 12:56:21 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001217067s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.XX................X.........X..........X.X........XXX......] duration: 29.103355516s trace: https://jaeger.dev.storj.io/trace/26b836a3b3565d7b FAIL (2026-01-08 12:56:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A) context deadline exceeded ======LOGS====== Jan 8 12:50:39.607666613 (813h21m39.478749648s) [upload=711444]: Starting upload Jan 8 12:50:39.607730554 (813h21m39.47881359s) [upload=711444]: Done starting upload Jan 8 12:50:39.607787793 (813h21m39.478870838s) [upload=711444]: Uploading segments... Jan 8 12:50:39.608018282 (813h21m39.479101317s) [upload=711444]: Got next segment. Inline: false Jan 8 12:50:39.608064500 (813h21m39.479147535s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:50:39.608071554 (813h21m39.479154599s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:50:39.855365281 (813h21m39.726448316s) [seg_pos={0 0}, upload=711444]: Begin upload segment... Jan 8 12:50:39.855411951 (813h21m39.726494986s) [seg_pos={0 0}, upload=711444]: Done begin upload segment. Jan 8 12:50:39.855442399 (813h21m39.726525434s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.855470101 (813h21m39.726553146s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.856865128 (813h21m39.727948173s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857139299 (813h21m39.728222334s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857486460 (813h21m39.728569505s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857515265 (813h21m39.72859831s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857951176 (813h21m39.729034221s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858247690 (813h21m39.729330735s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858350215 (813h21m39.72943326s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858712074 (813h21m39.729795109s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858998750 (813h21m39.730081785s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859272100 (813h21m39.730355145s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859598131 (813h21m39.730681167s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860030134 (813h21m39.731113169s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860557327 (813h21m39.731640372s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.861196736 (813h21m39.732279781s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:41.561070126 (813h21m41.432153171s) [upload=711444]: Next returned nil segment Jan 8 12:50:41.561087178 (813h21m41.432170233s) [upload=711444]: Waiting for error group managing segments... Jan 8 12:50:41.841406352 (813h21m41.712489397s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844334767 (813h21m41.715417802s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844410542 (813h21m41.715493587s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.852470178 (813h21m41.723553223s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.861679804 (813h21m41.73276285s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.865244672 (813h21m41.736327717s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.867385428 (813h21m41.738468463s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.872253999 (813h21m41.743337044s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.154329145 (813h21m42.025412191s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.208169825 (813h21m42.07925287s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.306412992 (813h21m42.177496027s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.334789194 (813h21m42.205872229s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486959725 (813h21m42.35804277s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486983420 (813h21m42.358066465s) [seg_pos={0 0}, upload=711444]: Segment reached optimal threshold of 13 pieces. Jan 8 12:50:42.487197378 (813h21m42.358280413s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection Jan 8 12:50:42.487205593 (813h21m42.358288628s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.487380176 (813h21m42.358463231s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:42.487387640 (813h21m42.358470685s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488052445 (813h21m42.359135481s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled Jan 8 12:50:42.488071081 (813h21m42.359154117s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488126787 (813h21m42.359209832s) [seg_pos={0 0}, upload=711444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:50:42.488234492 (813h21m42.359317537s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:50:42.488241265 (813h21m42.3593243s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:50:42.818540616 (813h21m42.689623651s) [upload=711444]: Done uploading segments... Jan 8 12:50:42.818557178 (813h21m42.689640213s) [upload=711444]: Upload finished. err: <nil> Duration: 29.103384421s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [XXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXXXXXXX.] duration: 3.692345308s trace: https://jaeger.dev.storj.io/trace/456b3785f58439c9 FAIL (2026-01-08 12:56:07 UTC) orbiter warning: context deadline exceeded Duration: 3.692371728s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...........X..X........X......X.X..............X...........X] duration: 30.000898186s trace: https://jaeger.dev.storj.io/trace/109d1a9d992c9104 FAIL (2026-01-08 12:55:34 UTC) orbiter warning: context deadline exceeded Duration: 30.00093748s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XX................X.........X..........X.X........XXX.....] duration: 5.80953008s trace: https://jaeger.dev.storj.io/trace/26b836a3b3565d7b FAIL (2026-01-08 12:55:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A) context deadline exceeded ======LOGS====== Jan 8 12:50:39.607666613 (813h21m39.478749648s) [upload=711444]: Starting upload Jan 8 12:50:39.607730554 (813h21m39.47881359s) [upload=711444]: Done starting upload Jan 8 12:50:39.607787793 (813h21m39.478870838s) [upload=711444]: Uploading segments... Jan 8 12:50:39.608018282 (813h21m39.479101317s) [upload=711444]: Got next segment. Inline: false Jan 8 12:50:39.608064500 (813h21m39.479147535s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:50:39.608071554 (813h21m39.479154599s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:50:39.855365281 (813h21m39.726448316s) [seg_pos={0 0}, upload=711444]: Begin upload segment... Jan 8 12:50:39.855411951 (813h21m39.726494986s) [seg_pos={0 0}, upload=711444]: Done begin upload segment. Jan 8 12:50:39.855442399 (813h21m39.726525434s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.855470101 (813h21m39.726553146s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.856865128 (813h21m39.727948173s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857139299 (813h21m39.728222334s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857486460 (813h21m39.728569505s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857515265 (813h21m39.72859831s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857951176 (813h21m39.729034221s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858247690 (813h21m39.729330735s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858350215 (813h21m39.72943326s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858712074 (813h21m39.729795109s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858998750 (813h21m39.730081785s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859272100 (813h21m39.730355145s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859598131 (813h21m39.730681167s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860030134 (813h21m39.731113169s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860557327 (813h21m39.731640372s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.861196736 (813h21m39.732279781s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:41.561070126 (813h21m41.432153171s) [upload=711444]: Next returned nil segment Jan 8 12:50:41.561087178 (813h21m41.432170233s) [upload=711444]: Waiting for error group managing segments... Jan 8 12:50:41.841406352 (813h21m41.712489397s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844334767 (813h21m41.715417802s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844410542 (813h21m41.715493587s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.852470178 (813h21m41.723553223s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.861679804 (813h21m41.73276285s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.865244672 (813h21m41.736327717s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.867385428 (813h21m41.738468463s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.872253999 (813h21m41.743337044s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.154329145 (813h21m42.025412191s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.208169825 (813h21m42.07925287s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.306412992 (813h21m42.177496027s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.334789194 (813h21m42.205872229s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486959725 (813h21m42.35804277s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486983420 (813h21m42.358066465s) [seg_pos={0 0}, upload=711444]: Segment reached optimal threshold of 13 pieces. Jan 8 12:50:42.487197378 (813h21m42.358280413s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection Jan 8 12:50:42.487205593 (813h21m42.358288628s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.487380176 (813h21m42.358463231s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:42.487387640 (813h21m42.358470685s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488052445 (813h21m42.359135481s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled Jan 8 12:50:42.488071081 (813h21m42.359154117s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488126787 (813h21m42.359209832s) [seg_pos={0 0}, upload=711444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:50:42.488234492 (813h21m42.359317537s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:50:42.488241265 (813h21m42.3593243s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:50:42.818540616 (813h21m42.689623651s) [upload=711444]: Done uploading segments... Jan 8 12:50:42.818557178 (813h21m42.689640213s) [upload=711444]: Upload finished. err: <nil> Duration: 5.809559245s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..................X.......X.............................X...] duration: 12.204106845s trace: https://jaeger.dev.storj.io/trace/766f5db5f644cf99 FAIL (2026-01-08 12:55:04 UTC) orbiter warning: context deadline exceeded Duration: 12.204126723s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...XX................X.........X..........X.X........XXX....] duration: 9.20951973s trace: https://jaeger.dev.storj.io/trace/26b836a3b3565d7b FAIL (2026-01-08 12:55:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A) context deadline exceeded ======LOGS====== Jan 8 12:50:39.607666613 (813h21m39.478749648s) [upload=711444]: Starting upload Jan 8 12:50:39.607730554 (813h21m39.47881359s) [upload=711444]: Done starting upload Jan 8 12:50:39.607787793 (813h21m39.478870838s) [upload=711444]: Uploading segments... Jan 8 12:50:39.608018282 (813h21m39.479101317s) [upload=711444]: Got next segment. Inline: false Jan 8 12:50:39.608064500 (813h21m39.479147535s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:50:39.608071554 (813h21m39.479154599s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:50:39.855365281 (813h21m39.726448316s) [seg_pos={0 0}, upload=711444]: Begin upload segment... Jan 8 12:50:39.855411951 (813h21m39.726494986s) [seg_pos={0 0}, upload=711444]: Done begin upload segment. Jan 8 12:50:39.855442399 (813h21m39.726525434s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.855470101 (813h21m39.726553146s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.856865128 (813h21m39.727948173s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857139299 (813h21m39.728222334s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857486460 (813h21m39.728569505s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857515265 (813h21m39.72859831s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857951176 (813h21m39.729034221s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858247690 (813h21m39.729330735s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858350215 (813h21m39.72943326s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858712074 (813h21m39.729795109s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858998750 (813h21m39.730081785s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859272100 (813h21m39.730355145s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859598131 (813h21m39.730681167s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860030134 (813h21m39.731113169s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860557327 (813h21m39.731640372s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.861196736 (813h21m39.732279781s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:41.561070126 (813h21m41.432153171s) [upload=711444]: Next returned nil segment Jan 8 12:50:41.561087178 (813h21m41.432170233s) [upload=711444]: Waiting for error group managing segments... Jan 8 12:50:41.841406352 (813h21m41.712489397s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844334767 (813h21m41.715417802s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844410542 (813h21m41.715493587s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.852470178 (813h21m41.723553223s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.861679804 (813h21m41.73276285s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.865244672 (813h21m41.736327717s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.867385428 (813h21m41.738468463s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.872253999 (813h21m41.743337044s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.154329145 (813h21m42.025412191s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.208169825 (813h21m42.07925287s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.306412992 (813h21m42.177496027s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.334789194 (813h21m42.205872229s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486959725 (813h21m42.35804277s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486983420 (813h21m42.358066465s) [seg_pos={0 0}, upload=711444]: Segment reached optimal threshold of 13 pieces. Jan 8 12:50:42.487197378 (813h21m42.358280413s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection Jan 8 12:50:42.487205593 (813h21m42.358288628s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.487380176 (813h21m42.358463231s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:42.487387640 (813h21m42.358470685s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488052445 (813h21m42.359135481s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled Jan 8 12:50:42.488071081 (813h21m42.359154117s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488126787 (813h21m42.359209832s) [seg_pos={0 0}, upload=711444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:50:42.488234492 (813h21m42.359317537s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:50:42.488241265 (813h21m42.3593243s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:50:42.818540616 (813h21m42.689623651s) [upload=711444]: Done uploading segments... Jan 8 12:50:42.818557178 (813h21m42.689640213s) [upload=711444]: Upload finished. err: <nil> Duration: 9.209547514s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXXXXXXX] duration: 5.000750138s trace: https://jaeger.dev.storj.io/trace/456b3785f58439c9 FAIL (2026-01-08 12:54:57 UTC) orbiter warning: context deadline exceeded Duration: 5.000774254s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............X..X........X......X.X..............X...........] duration: 18.963266229s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:54:44 UTC) orbiter warning: context deadline exceeded Duration: 18.963318187s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXXXXXX] duration: 5.000568292s trace: https://jaeger.dev.storj.io/trace/2b9b3cca6a056b9c FAIL (2026-01-08 12:54:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000590945s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...................X.......X.............................X..] duration: 27.481571848s trace: https://jaeger.dev.storj.io/trace/766f5db5f644cf99 FAIL (2026-01-08 12:54:33 UTC) orbiter warning: context deadline exceeded Duration: 27.481584171s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....................X.......X.............................X.] duration: 25.078997961s trace: https://jaeger.dev.storj.io/trace/766f5db5f644cf99 FAIL (2026-01-08 12:54:05 UTC) orbiter warning: context deadline exceeded Duration: 25.079008272s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [....XX................X.........X..........X.X........XXX...] duration: 6.223610213s trace: https://jaeger.dev.storj.io/trace/26b836a3b3565d7b FAIL (2026-01-08 12:53:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A) context deadline exceeded ======LOGS====== Jan 8 12:50:39.607666613 (813h21m39.478749648s) [upload=711444]: Starting upload Jan 8 12:50:39.607730554 (813h21m39.47881359s) [upload=711444]: Done starting upload Jan 8 12:50:39.607787793 (813h21m39.478870838s) [upload=711444]: Uploading segments... Jan 8 12:50:39.608018282 (813h21m39.479101317s) [upload=711444]: Got next segment. Inline: false Jan 8 12:50:39.608064500 (813h21m39.479147535s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:50:39.608071554 (813h21m39.479154599s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:50:39.855365281 (813h21m39.726448316s) [seg_pos={0 0}, upload=711444]: Begin upload segment... Jan 8 12:50:39.855411951 (813h21m39.726494986s) [seg_pos={0 0}, upload=711444]: Done begin upload segment. Jan 8 12:50:39.855442399 (813h21m39.726525434s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.855470101 (813h21m39.726553146s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.856865128 (813h21m39.727948173s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857139299 (813h21m39.728222334s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857486460 (813h21m39.728569505s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857515265 (813h21m39.72859831s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857951176 (813h21m39.729034221s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858247690 (813h21m39.729330735s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858350215 (813h21m39.72943326s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858712074 (813h21m39.729795109s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858998750 (813h21m39.730081785s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859272100 (813h21m39.730355145s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859598131 (813h21m39.730681167s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860030134 (813h21m39.731113169s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860557327 (813h21m39.731640372s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.861196736 (813h21m39.732279781s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:41.561070126 (813h21m41.432153171s) [upload=711444]: Next returned nil segment Jan 8 12:50:41.561087178 (813h21m41.432170233s) [upload=711444]: Waiting for error group managing segments... Jan 8 12:50:41.841406352 (813h21m41.712489397s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844334767 (813h21m41.715417802s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844410542 (813h21m41.715493587s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.852470178 (813h21m41.723553223s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.861679804 (813h21m41.73276285s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.865244672 (813h21m41.736327717s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.867385428 (813h21m41.738468463s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.872253999 (813h21m41.743337044s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.154329145 (813h21m42.025412191s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.208169825 (813h21m42.07925287s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.306412992 (813h21m42.177496027s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.334789194 (813h21m42.205872229s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486959725 (813h21m42.35804277s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486983420 (813h21m42.358066465s) [seg_pos={0 0}, upload=711444]: Segment reached optimal threshold of 13 pieces. Jan 8 12:50:42.487197378 (813h21m42.358280413s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection Jan 8 12:50:42.487205593 (813h21m42.358288628s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.487380176 (813h21m42.358463231s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:42.487387640 (813h21m42.358470685s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488052445 (813h21m42.359135481s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled Jan 8 12:50:42.488071081 (813h21m42.359154117s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488126787 (813h21m42.359209832s) [seg_pos={0 0}, upload=711444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:50:42.488234492 (813h21m42.359317537s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:50:42.488241265 (813h21m42.3593243s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:50:42.818540616 (813h21m42.689623651s) [upload=711444]: Done uploading segments... Jan 8 12:50:42.818557178 (813h21m42.689640213s) [upload=711444]: Upload finished. err: <nil> Duration: 6.22364038s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............X..X........X......X.X..............X..........] duration: 22.110341188s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:53:44 UTC) orbiter warning: context deadline exceeded Duration: 22.110360224s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....................X.......X.............................X] duration: 1m0.001111003s trace: https://jaeger.dev.storj.io/trace/766f5db5f644cf99 FAIL (2026-01-08 12:53:40 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001124789s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXXXXX] duration: 5.000480689s trace: https://jaeger.dev.storj.io/trace/44636abf0939853f FAIL (2026-01-08 12:53:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000505486s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............X..X........X......X.X..............X.........] duration: 18.898623858s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:52:50 UTC) orbiter warning: context deadline exceeded Duration: 18.898652904s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....XX................X.........X..........X.X........XXX..] duration: 6.200090003s trace: https://jaeger.dev.storj.io/trace/26b836a3b3565d7b FAIL (2026-01-08 12:52:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A) context deadline exceeded ======LOGS====== Jan 8 12:50:39.607666613 (813h21m39.478749648s) [upload=711444]: Starting upload Jan 8 12:50:39.607730554 (813h21m39.47881359s) [upload=711444]: Done starting upload Jan 8 12:50:39.607787793 (813h21m39.478870838s) [upload=711444]: Uploading segments... Jan 8 12:50:39.608018282 (813h21m39.479101317s) [upload=711444]: Got next segment. Inline: false Jan 8 12:50:39.608064500 (813h21m39.479147535s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:50:39.608071554 (813h21m39.479154599s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:50:39.855365281 (813h21m39.726448316s) [seg_pos={0 0}, upload=711444]: Begin upload segment... Jan 8 12:50:39.855411951 (813h21m39.726494986s) [seg_pos={0 0}, upload=711444]: Done begin upload segment. Jan 8 12:50:39.855442399 (813h21m39.726525434s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.855470101 (813h21m39.726553146s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.856865128 (813h21m39.727948173s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857139299 (813h21m39.728222334s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857486460 (813h21m39.728569505s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857515265 (813h21m39.72859831s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857951176 (813h21m39.729034221s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858247690 (813h21m39.729330735s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858350215 (813h21m39.72943326s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858712074 (813h21m39.729795109s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858998750 (813h21m39.730081785s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859272100 (813h21m39.730355145s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859598131 (813h21m39.730681167s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860030134 (813h21m39.731113169s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860557327 (813h21m39.731640372s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.861196736 (813h21m39.732279781s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:41.561070126 (813h21m41.432153171s) [upload=711444]: Next returned nil segment Jan 8 12:50:41.561087178 (813h21m41.432170233s) [upload=711444]: Waiting for error group managing segments... Jan 8 12:50:41.841406352 (813h21m41.712489397s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844334767 (813h21m41.715417802s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844410542 (813h21m41.715493587s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.852470178 (813h21m41.723553223s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.861679804 (813h21m41.73276285s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.865244672 (813h21m41.736327717s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.867385428 (813h21m41.738468463s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.872253999 (813h21m41.743337044s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.154329145 (813h21m42.025412191s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.208169825 (813h21m42.07925287s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.306412992 (813h21m42.177496027s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.334789194 (813h21m42.205872229s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486959725 (813h21m42.35804277s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486983420 (813h21m42.358066465s) [seg_pos={0 0}, upload=711444]: Segment reached optimal threshold of 13 pieces. Jan 8 12:50:42.487197378 (813h21m42.358280413s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection Jan 8 12:50:42.487205593 (813h21m42.358288628s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.487380176 (813h21m42.358463231s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:42.487387640 (813h21m42.358470685s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488052445 (813h21m42.359135481s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled Jan 8 12:50:42.488071081 (813h21m42.359154117s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488126787 (813h21m42.359209832s) [seg_pos={0 0}, upload=711444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:50:42.488234492 (813h21m42.359317537s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:50:42.488241265 (813h21m42.3593243s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:50:42.818540616 (813h21m42.689623651s) [upload=711444]: Done uploading segments... Jan 8 12:50:42.818557178 (813h21m42.689640213s) [upload=711444]: Upload finished. err: <nil> Duration: 6.200115581s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............X..X........X......X.X..............X........] duration: 17.829697848s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:52:30 UTC) orbiter warning: context deadline exceeded Duration: 17.829747444s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXXXX] duration: 5.000425491s trace: https://jaeger.dev.storj.io/trace/5f630ab3a326e376 FAIL (2026-01-08 12:52:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000450308s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................X..X........X......X.X..............X.......] duration: 21.343805023s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:51:57 UTC) orbiter warning: context deadline exceeded Duration: 21.343831663s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................X.......X.............................] duration: 16.625791619s trace: https://jaeger.dev.storj.io/trace/f8dd51e1a9bb81f FAIL (2026-01-08 12:51:53 UTC) orbiter warning: context deadline exceeded Duration: 16.625820494s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [......XX................X.........X..........X.X........XXX.] duration: 5.749531983s trace: https://jaeger.dev.storj.io/trace/26b836a3b3565d7b FAIL (2026-01-08 12:51:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A) context deadline exceeded ======LOGS====== Jan 8 12:50:39.607666613 (813h21m39.478749648s) [upload=711444]: Starting upload Jan 8 12:50:39.607730554 (813h21m39.47881359s) [upload=711444]: Done starting upload Jan 8 12:50:39.607787793 (813h21m39.478870838s) [upload=711444]: Uploading segments... Jan 8 12:50:39.608018282 (813h21m39.479101317s) [upload=711444]: Got next segment. Inline: false Jan 8 12:50:39.608064500 (813h21m39.479147535s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:50:39.608071554 (813h21m39.479154599s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:50:39.855365281 (813h21m39.726448316s) [seg_pos={0 0}, upload=711444]: Begin upload segment... Jan 8 12:50:39.855411951 (813h21m39.726494986s) [seg_pos={0 0}, upload=711444]: Done begin upload segment. Jan 8 12:50:39.855442399 (813h21m39.726525434s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.855470101 (813h21m39.726553146s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.856865128 (813h21m39.727948173s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857139299 (813h21m39.728222334s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857486460 (813h21m39.728569505s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857515265 (813h21m39.72859831s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857951176 (813h21m39.729034221s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858247690 (813h21m39.729330735s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858350215 (813h21m39.72943326s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858712074 (813h21m39.729795109s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858998750 (813h21m39.730081785s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859272100 (813h21m39.730355145s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859598131 (813h21m39.730681167s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860030134 (813h21m39.731113169s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860557327 (813h21m39.731640372s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.861196736 (813h21m39.732279781s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:41.561070126 (813h21m41.432153171s) [upload=711444]: Next returned nil segment Jan 8 12:50:41.561087178 (813h21m41.432170233s) [upload=711444]: Waiting for error group managing segments... Jan 8 12:50:41.841406352 (813h21m41.712489397s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844334767 (813h21m41.715417802s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844410542 (813h21m41.715493587s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.852470178 (813h21m41.723553223s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.861679804 (813h21m41.73276285s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.865244672 (813h21m41.736327717s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.867385428 (813h21m41.738468463s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.872253999 (813h21m41.743337044s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.154329145 (813h21m42.025412191s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.208169825 (813h21m42.07925287s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.306412992 (813h21m42.177496027s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.334789194 (813h21m42.205872229s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486959725 (813h21m42.35804277s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486983420 (813h21m42.358066465s) [seg_pos={0 0}, upload=711444]: Segment reached optimal threshold of 13 pieces. Jan 8 12:50:42.487197378 (813h21m42.358280413s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection Jan 8 12:50:42.487205593 (813h21m42.358288628s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.487380176 (813h21m42.358463231s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:42.487387640 (813h21m42.358470685s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488052445 (813h21m42.359135481s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled Jan 8 12:50:42.488071081 (813h21m42.359154117s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488126787 (813h21m42.359209832s) [seg_pos={0 0}, upload=711444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:50:42.488234492 (813h21m42.359317537s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:50:42.488241265 (813h21m42.3593243s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:50:42.818540616 (813h21m42.689623651s) [upload=711444]: Done uploading segments... Jan 8 12:50:42.818557178 (813h21m42.689640213s) [upload=711444]: Upload finished. err: <nil> Duration: 5.749571788s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXXX] duration: 5.001341069s trace: https://jaeger.dev.storj.io/trace/3e73b194165c5606 FAIL (2026-01-08 12:51:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001367859s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.......XX................X.........X..........X.X........XXX] duration: 30.005512987s trace: https://jaeger.dev.storj.io/trace/26b836a3b3565d7b FAIL (2026-01-08 12:51:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, Piece ID: H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA) context deadline exceeded; piecestore: (Node ID: 12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, Piece ID: OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ) context deadline exceeded; piecestore: (Node ID: 1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, Piece ID: CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A) context deadline exceeded ======LOGS====== Jan 8 12:50:39.607666613 (813h21m39.478749648s) [upload=711444]: Starting upload Jan 8 12:50:39.607730554 (813h21m39.47881359s) [upload=711444]: Done starting upload Jan 8 12:50:39.607787793 (813h21m39.478870838s) [upload=711444]: Uploading segments... Jan 8 12:50:39.608018282 (813h21m39.479101317s) [upload=711444]: Got next segment. Inline: false Jan 8 12:50:39.608064500 (813h21m39.479147535s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:50:39.608071554 (813h21m39.479154599s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:50:39.855365281 (813h21m39.726448316s) [seg_pos={0 0}, upload=711444]: Begin upload segment... Jan 8 12:50:39.855411951 (813h21m39.726494986s) [seg_pos={0 0}, upload=711444]: Done begin upload segment. Jan 8 12:50:39.855442399 (813h21m39.726525434s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.855470101 (813h21m39.726553146s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.856865128 (813h21m39.727948173s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857139299 (813h21m39.728222334s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857486460 (813h21m39.728569505s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857515265 (813h21m39.72859831s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.857951176 (813h21m39.729034221s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858247690 (813h21m39.729330735s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858350215 (813h21m39.72943326s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858712074 (813h21m39.729795109s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.858998750 (813h21m39.730081785s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859272100 (813h21m39.730355145s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.859598131 (813h21m39.730681167s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860030134 (813h21m39.731113169s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.860557327 (813h21m39.731640372s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:39.861196736 (813h21m39.732279781s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Uploading piece... Jan 8 12:50:41.561070126 (813h21m41.432153171s) [upload=711444]: Next returned nil segment Jan 8 12:50:41.561087178 (813h21m41.432170233s) [upload=711444]: Waiting for error group managing segments... Jan 8 12:50:41.841406352 (813h21m41.712489397s) [noise=true , address=145.79.227.16:4909 , piece_id=FUHPORFYIC7TTFDI23VPI5NLZNM7YZFWHGJBOONMEUF6NQYCVRUA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844334767 (813h21m41.715417802s) [noise=true , address=145.79.227.7:4294 , piece_id=OECAIEZFSKVK24UBFXNMJWHHOPALMEXAPJS3QK7K4YGAVWPBBIIA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.844410542 (813h21m41.715493587s) [noise=true , address=145.79.227.48:4303 , piece_id=77SWWGBPFK7RNJ4BBFXYVMIU63YKNG3UVQPH2SWISSMC56SRPRKQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.852470178 (813h21m41.723553223s) [noise=true , address=145.79.227.39:4014 , piece_id=MJ7ISP5RVCBXIOWUAY2NRXVYEDIEBIVFL3PGUBFTFXMJJMMB2ZMQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.861679804 (813h21m41.73276285s) [noise=true , address=145.79.227.29:4941 , piece_id=SO2YRQION542EBBLMY6KWYQFBA6KFOG3IOE5Y4A4JYOMQQYG4OWQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.865244672 (813h21m41.736327717s) [noise=true , address=145.79.227.46:4603 , piece_id=GVKTXPQY34FHQ4ZTAZO5OWZWOU663SDZ746GWQSX57KMNZFMBMTA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.867385428 (813h21m41.738468463s) [noise=true , address=145.79.227.5:4223 , piece_id=3YHM6SB73QXCJX2V75U26WJWO6U44AZG5MDI6IDCUMRS4X7MYS2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:41.872253999 (813h21m41.743337044s) [noise=true , address=145.79.227.25:4618 , piece_id=DZXZTHAYXFI2FGPH3BNNQMJTRXXWHSYEF3EPTV7XC5IZ47RPFSCA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.154329145 (813h21m42.025412191s) [noise=true , address=77.90.31.11:4555 , piece_id=A3BYYRNUHGPJSVXHJTSQIIZXYV75OSNW2HI227GUWZBJ6RXIWW2A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.208169825 (813h21m42.07925287s) [noise=true , address=77.90.31.14:4661 , piece_id=OVEVOK6K4BWX7AZVVUGCC4CGJTSOOIVLF24H6NY37PTBNQUJ7IEQ, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.306412992 (813h21m42.177496027s) [noise=true , address=45.91.116.14:4191 , piece_id=H4A23J2N575JVW7I3DEJWJ2BBM6RYORG7LVKI5C7XGW5D2OHT2WA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.334789194 (813h21m42.205872229s) [noise=true , address=45.91.116.9:4262 , piece_id=JAJLQKRPNNXJS7F6QGV4ZA2XJVI442SJLN77UFLHREVZ7STHVURA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486959725 (813h21m42.35804277s) [noise=true , address=77.90.31.13:4627 , piece_id=CGFICUY4D5KLSOAVCW7FTRYCQJA2R4PBIJKFOS4HSIFZ33AY3Z7A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: <nil> Jan 8 12:50:42.486983420 (813h21m42.358066465s) [seg_pos={0 0}, upload=711444]: Segment reached optimal threshold of 13 pieces. Jan 8 12:50:42.487197378 (813h21m42.358280413s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t8YAgwwGdjWLFyb9yNgLSFXV8QHbdyv2tnjtSZQAcBc8cZiHq): protocol: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34890->45.91.116.5:4300: use of closed network connection Jan 8 12:50:42.487205593 (813h21m42.358288628s) [noise=true , address=45.91.116.5:4300 , piece_id=PTMRLZXDS5LHXE3KI5U7GNNOIRBG5GTKUMMVC4ONJBJCP2IYZ47A, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.487380176 (813h21m42.358463231s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:42.487387640 (813h21m42.358470685s) [noise=true , address=45.91.116.4:4937 , piece_id=ARRPKQBMQIJDLBLW5J5IL5Z5IAL5O7A5MZFIO7MJMELHT7DVMNPA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488052445 (813h21m42.359135481s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: context canceled; context canceled Jan 8 12:50:42.488071081 (813h21m42.359154117s) [noise=true , address=77.90.31.18:4805 , piece_id=RKCJHIICGW7AV5YY3SMCQY5DXTADQCSWHQHMEWUGHVVAP4MIN5PA, seg_pos={0 0}, upload=711444]: Long tail cancellation due to optimal threshold Jan 8 12:50:42.488126787 (813h21m42.359209832s) [seg_pos={0 0}, upload=711444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:50:42.488234492 (813h21m42.359317537s) [upload=711444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:50:42.488241265 (813h21m42.3593243s) [upload=711444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:50:42.818540616 (813h21m42.689623651s) [upload=711444]: Done uploading segments... Jan 8 12:50:42.818557178 (813h21m42.689640213s) [upload=711444]: Upload finished. err: <nil> Duration: 30.00556664s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................X..X........X......X.X..............X......] duration: 27.907686603s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:51:00 UTC) orbiter warning: context deadline exceeded Duration: 27.907712903s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.......X............................] duration: 10.166106523s trace: https://jaeger.dev.storj.io/trace/f8dd51e1a9bb81f FAIL (2026-01-08 12:50:57 UTC) orbiter warning: context deadline exceeded Duration: 10.166169103s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXXX] duration: 5.001247828s trace: https://jaeger.dev.storj.io/trace/25c43b2dd35c151e FAIL (2026-01-08 12:50:50 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001287323s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [........XX................X.........X..........X.X........XX] duration: 30.003395083s trace: https://jaeger.dev.storj.io/trace/44a50ba15bffdb68 FAIL (2026-01-08 12:50:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, Piece ID: QGCI3VOTFGPTHDIT253NCUO5GWOZS62TKIJ34CUDGJGTPG76B3QQ) context deadline exceeded; piecestore: (Node ID: 12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, Piece ID: PPIZEKNDBQOKYASPFONRCLXR3ZAINABZNOZI4H4BTJW4U3ZK5LBA) context deadline exceeded ======LOGS====== Jan 8 12:50:09.380832103 (813h21m9.251915138s) [upload=711433]: Starting upload Jan 8 12:50:09.380889943 (813h21m9.251972988s) [upload=711433]: Done starting upload Jan 8 12:50:09.381089854 (813h21m9.252172899s) [upload=711433]: Uploading segments... Jan 8 12:50:09.381131273 (813h21m9.252214308s) [upload=711433]: Got next segment. Inline: false Jan 8 12:50:09.381164065 (813h21m9.252247111s) [upload=711433]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:50:09.381169405 (813h21m9.252252441s) [upload=711433]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:50:09.627171274 (813h21m9.498254309s) [seg_pos={0 0}, upload=711433]: Begin upload segment... Jan 8 12:50:09.627218543 (813h21m9.498301588s) [seg_pos={0 0}, upload=711433]: Done begin upload segment. Jan 8 12:50:09.627244092 (813h21m9.498327137s) [noise=true , address=145.79.227.48:4302 , piece_id=ZE6I7IXGDCC62ETZC23JAQEYCYVZLRNLYGJGDVQWNH2B3PKA2NDQ, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.627361966 (813h21m9.498445021s) [noise=true , address=145.79.227.14:4837 , piece_id=IOBEZDTQBH5TQHLJBALJM7QXHHPKBUTA5MHMDFP5NOBZ5SD6JI6Q, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.627955096 (813h21m9.499038141s) [noise=true , address=45.91.116.4:4939 , piece_id=HSWUGI4NB75AAM3FPRWR4U3PSLLCYG5LZZAWEU7MFPKAQE5YBWBA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.628113818 (813h21m9.499196863s) [noise=true , address=145.79.227.47:4267 , piece_id=U6QN6CXJHEC37DK4EV34L3D2272K3AXRTZUZJLLQO6BP54XQARMA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.628346662 (813h21m9.499429697s) [noise=true , address=145.79.227.50:4375 , piece_id=WKQUTDUFPQ6FUXLIMTM7EXNC2EPD76IW7B2BRDXLKE3JHEOZU5RA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.628463374 (813h21m9.499546419s) [noise=true , address=145.79.227.52:4447 , piece_id=ATUCAB4STRVOIYVSH4L57TFT4QBKFZLBNVFWNAYTBJ6NSQQ6ZBRA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.628669797 (813h21m9.499752832s) [noise=true , address=45.91.116.8:4046 , piece_id=PWJASXHH2DIGVGJTIZC56YAY6EUVC33SFHCV6BB6LYMJTZLNRY5A, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.628818550 (813h21m9.499901595s) [noise=true , address=145.79.227.2:4113 , piece_id=YCBY2V4FTYM3KKC2DPUISJXXSFSSHRBBSGICTMOM2EKXYM36UZXQ, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.628946995 (813h21m9.50003004s) [noise=true , address=77.90.31.19:4842 , piece_id=QGCI3VOTFGPTHDIT253NCUO5GWOZS62TKIJ34CUDGJGTPG76B3QQ, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.629198534 (813h21m9.500281579s) [noise=true , address=145.79.227.16:4912 , piece_id=J4AKHPUH5DDNAEXLNUUKY4L5SP6UYN5KE6ZFYG5G4JSZOTWLL6WA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.629225766 (813h21m9.500308801s) [noise=true , address=45.91.116.10:4084 , piece_id=LIZUEBVVECJ3UK4W43NKFATA22763JT62HZRUNMXDVTTL37HAGEA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.629587244 (813h21m9.500670279s) [noise=true , address=145.79.227.42:4123 , piece_id=OUBTIO2MQKLH74KJRAAK4HJ7Z4LRQ2QVY4JQ3P565A4BWVGIGDSQ, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.629671334 (813h21m9.500754369s) [noise=true , address=145.79.227.21:4474 , piece_id=6X2JDS57HKA7LL6TMQOAIWYH3YHH2WMGIJLDZEVMXBXIJCEMSCXA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.630076966 (813h21m9.501160011s) [noise=true , address=145.79.227.4:4186 , piece_id=AN54WSMO4AYXT664ZFN2DRO7CADEAQGITPXHUFA2LBON3GNLJNXA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.630188630 (813h21m9.501271675s) [noise=true , address=77.90.31.21:4878 , piece_id=PPIZEKNDBQOKYASPFONRCLXR3ZAINABZNOZI4H4BTJW4U3ZK5LBA, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:09.630935622 (813h21m9.502018697s) [noise=true , address=145.79.227.7:4295 , piece_id=HLCPZ3HVPF4DLA2TGSBCNCQOPGERAONYMGOKRLNHHODVKQBAAKAQ, seg_pos={0 0}, upload=711433]: Uploading piece... Jan 8 12:50:11.335538998 (813h21m11.206622043s) [upload=711433]: Next returned nil segment Jan 8 12:50:11.335571229 (813h21m11.206654295s) [upload=711433]: Waiting for error group managing segments... Jan 8 12:50:11.618261296 (813h21m11.489344342s) [noise=true , address=145.79.227.48:4302 , piece_id=ZE6I7IXGDCC62ETZC23JAQEYCYVZLRNLYGJGDVQWNH2B3PKA2NDQ, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.623787278 (813h21m11.494870323s) [noise=true , address=145.79.227.21:4474 , piece_id=6X2JDS57HKA7LL6TMQOAIWYH3YHH2WMGIJLDZEVMXBXIJCEMSCXA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.632491172 (813h21m11.503574217s) [noise=true , address=145.79.227.2:4113 , piece_id=YCBY2V4FTYM3KKC2DPUISJXXSFSSHRBBSGICTMOM2EKXYM36UZXQ, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.634631889 (813h21m11.505714934s) [noise=true , address=145.79.227.52:4447 , piece_id=ATUCAB4STRVOIYVSH4L57TFT4QBKFZLBNVFWNAYTBJ6NSQQ6ZBRA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.636376350 (813h21m11.507459395s) [noise=true , address=145.79.227.7:4295 , piece_id=HLCPZ3HVPF4DLA2TGSBCNCQOPGERAONYMGOKRLNHHODVKQBAAKAQ, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.636744390 (813h21m11.507827426s) [noise=true , address=145.79.227.16:4912 , piece_id=J4AKHPUH5DDNAEXLNUUKY4L5SP6UYN5KE6ZFYG5G4JSZOTWLL6WA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.646776663 (813h21m11.517859709s) [noise=true , address=145.79.227.14:4837 , piece_id=IOBEZDTQBH5TQHLJBALJM7QXHHPKBUTA5MHMDFP5NOBZ5SD6JI6Q, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.653792001 (813h21m11.524875056s) [noise=true , address=145.79.227.47:4267 , piece_id=U6QN6CXJHEC37DK4EV34L3D2272K3AXRTZUZJLLQO6BP54XQARMA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.667138836 (813h21m11.538221881s) [noise=true , address=145.79.227.50:4375 , piece_id=WKQUTDUFPQ6FUXLIMTM7EXNC2EPD76IW7B2BRDXLKE3JHEOZU5RA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.683278557 (813h21m11.554361602s) [noise=true , address=145.79.227.42:4123 , piece_id=OUBTIO2MQKLH74KJRAAK4HJ7Z4LRQ2QVY4JQ3P565A4BWVGIGDSQ, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.713023977 (813h21m11.584107022s) [noise=true , address=145.79.227.4:4186 , piece_id=AN54WSMO4AYXT664ZFN2DRO7CADEAQGITPXHUFA2LBON3GNLJNXA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.947893707 (813h21m11.818976752s) [noise=true , address=77.90.31.19:4842 , piece_id=QGCI3VOTFGPTHDIT253NCUO5GWOZS62TKIJ34CUDGJGTPG76B3QQ, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.987042121 (813h21m11.858125156s) [noise=true , address=77.90.31.21:4878 , piece_id=PPIZEKNDBQOKYASPFONRCLXR3ZAINABZNOZI4H4BTJW4U3ZK5LBA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: <nil> Jan 8 12:50:11.987065395 (813h21m11.85814843s) [seg_pos={0 0}, upload=711433]: Segment reached optimal threshold of 13 pieces. Jan 8 12:50:11.987314560 (813h21m11.858397605s) [noise=true , address=45.91.116.8:4046 , piece_id=PWJASXHH2DIGVGJTIZC56YAY6EUVC33SFHCV6BB6LYMJTZLNRY5A, seg_pos={0 0}, upload=711433]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:11.987323356 (813h21m11.858406391s) [noise=true , address=45.91.116.8:4046 , piece_id=PWJASXHH2DIGVGJTIZC56YAY6EUVC33SFHCV6BB6LYMJTZLNRY5A, seg_pos={0 0}, upload=711433]: Long tail cancellation due to optimal threshold Jan 8 12:50:11.987368843 (813h21m11.858451888s) [noise=true , address=45.91.116.4:4939 , piece_id=HSWUGI4NB75AAM3FPRWR4U3PSLLCYG5LZZAWEU7MFPKAQE5YBWBA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:46052->45.91.116.4:4939: use of closed network connection; write tcp 188.166.225.170:46052->45.91.116.4:4939: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46052->45.91.116.4:4939: use of closed network connection Jan 8 12:50:11.987385595 (813h21m11.85846864s) [noise=true , address=45.91.116.4:4939 , piece_id=HSWUGI4NB75AAM3FPRWR4U3PSLLCYG5LZZAWEU7MFPKAQE5YBWBA, seg_pos={0 0}, upload=711433]: Long tail cancellation due to optimal threshold Jan 8 12:50:11.987616494 (813h21m11.858699549s) [noise=true , address=45.91.116.10:4084 , piece_id=LIZUEBVVECJ3UK4W43NKFATA22763JT62HZRUNMXDVTTL37HAGEA, seg_pos={0 0}, upload=711433]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj): protocol: expected piece hash; context canceled; EOF Jan 8 12:50:11.987625332 (813h21m11.858708377s) [noise=true , address=45.91.116.10:4084 , piece_id=LIZUEBVVECJ3UK4W43NKFATA22763JT62HZRUNMXDVTTL37HAGEA, seg_pos={0 0}, upload=711433]: Long tail cancellation due to optimal threshold Jan 8 12:50:11.987644047 (813h21m11.858727082s) [seg_pos={0 0}, upload=711433]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:50:11.987756421 (813h21m11.858839466s) [upload=711433]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:50:11.987762453 (813h21m11.858845498s) [upload=711433]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:50:12.291371571 (813h21m12.162454647s) [upload=711433]: Done uploading segments... Jan 8 12:50:12.291397010 (813h21m12.162480075s) [upload=711433]: Upload finished. err: <nil> Duration: 30.003415372s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................X.......X...........................] duration: 29.212454684s trace: https://jaeger.dev.storj.io/trace/f8dd51e1a9bb81f FAIL (2026-01-08 12:50:38 UTC) orbiter warning: context deadline exceeded Duration: 29.212467069s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................X..X........X......X.X..............X.....] duration: 27.669126495s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:50:30 UTC) orbiter warning: context deadline exceeded Duration: 27.669140431s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 54 out of 60 is failed (threshold=6) [XXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXXX] duration: 5.000520854s trace: https://jaeger.dev.storj.io/trace/4be0c11f2b713ae2 FAIL (2026-01-08 12:50:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000542214s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.......X..........................] duration: 28.941791564s trace: https://jaeger.dev.storj.io/trace/f8dd51e1a9bb81f FAIL (2026-01-08 12:50:08 UTC) orbiter warning: context deadline exceeded Duration: 28.941815218s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X........XX................X.........X..........X.X........X] duration: 30.004173376s trace: https://jaeger.dev.storj.io/trace/1498495cc20876e3 FAIL (2026-01-08 12:50:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Jan 8 12:49:35.158557060 (813h20m35.029640126s) [upload=711423]: Starting upload Jan 8 12:49:35.158637745 (813h20m35.02972081s) [upload=711423]: Done starting upload Jan 8 12:49:35.158727445 (813h20m35.02981049s) [upload=711423]: Uploading segments... Jan 8 12:49:35.158871559 (813h20m35.029954604s) [upload=711423]: Got next segment. Inline: false Jan 8 12:49:35.158902428 (813h20m35.029985493s) [upload=711423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:49:35.158910353 (813h20m35.029993388s) [upload=711423]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:49:35.420359274 (813h20m35.291442309s) [seg_pos={0 0}, upload=711423]: Begin upload segment... Jan 8 12:49:35.420416604 (813h20m35.291499639s) [seg_pos={0 0}, upload=711423]: Done begin upload segment. Jan 8 12:49:35.420454055 (813h20m35.2915371s) [noise=true , address=145.79.227.17:4972 , piece_id=HM5IX3CSC3YBEB65W2IQLYZALHKUMOGDYFTXG3GBJNUGBNHYQY2A, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.420513118 (813h20m35.291596163s) [noise=true , address=145.79.227.23:4546 , piece_id=NSSD6MR56ZCGB35XZWWVDJ3YLUUIMUSHZW643B2L2KQHYT7UVGSQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.420618859 (813h20m35.291701904s) [noise=true , address=77.90.31.15:4697 , piece_id=XFBYEWBTPC5MHBCVDOB7WRJL4CTBHTCJUIVCV3ZMBBLJWZFEDVGQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.421159378 (813h20m35.292242413s) [noise=true , address=145.79.227.39:4014 , piece_id=BRJSW66BJGLHP3YNWYQBFWNOXEO5DWUALF73GU6TXVXTBPTLFSOQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.421658729 (813h20m35.292741764s) [noise=true , address=145.79.227.48:4303 , piece_id=L3KCG3IXMUYNW3DYUQAPSJZMEIXO34GLY4CXVPYGCFWKWBJDFNZQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.422104919 (813h20m35.293187954s) [noise=true , address=145.79.227.41:4087 , piece_id=DLEOLC6JZR72IPZ5BXHB3DPW55RNA7YFBQKVVVDMD2GR63NELK6Q, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.422428455 (813h20m35.2935115s) [noise=true , address=77.90.31.18:4808 , piece_id=YYZP2EWLHVLPLBOWNP6V3WGMSTPQSKN6IMYSRSSHZ2QWKAV4C2ZA, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.422743795 (813h20m35.29382684s) [noise=true , address=77.90.31.14:4663 , piece_id=RKVXNWGN4633WKAWS2337N3OVBFXMHT4RP2VPGDOMRHDKXYLA3MA, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.423075476 (813h20m35.294158512s) [noise=true , address=145.79.227.12:4765 , piece_id=N3RV4OLEYXGDSBBZIGVUTHKBBTJPQ25OM5KS3QL2IGTJNKAAM3QQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.423448737 (813h20m35.294531773s) [noise=true , address=77.90.31.11:4553 , piece_id=YIJFVBDYRTRWBWLFS5BS23Q7FJ5P5SSVU4IO2E2OK3X5GGE77WEA, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.423884979 (813h20m35.294968034s) [noise=true , address=145.79.227.29:4943 , piece_id=5SLUJ5WZ5DKOWBFUBXGSGCGWSVXXQDCGPTZB5K2LAM4TJLELBMSQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.424370944 (813h20m35.295453989s) [noise=true , address=45.91.116.11:4119 , piece_id=U5LLYWUM5CDOB6522ZMEMXWYEP67XWNBWY3MTOUX4L4X3PKSMSUQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.424856228 (813h20m35.295939273s) [noise=true , address=145.79.227.30:4762 , piece_id=4627M3HYYIXI4SUKN2Z7HJ6UXOCDIGT63VVWLOCKRM23Q7WNB4OQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.425320552 (813h20m35.296403587s) [noise=true , address=45.91.116.6:4974 , piece_id=NDHADQKHSV2DTNQCMS563HAQBYYSESU4RAQMCW3PJRG6HJZYQQZA, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.425767964 (813h20m35.296851009s) [noise=true , address=145.79.227.50:4373 , piece_id=I3JJJBTRRC22BWKWR6KQIVDBEUF7HGJAHI2VHPRHOPQ5SSW2DGZQ, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:35.426104114 (813h20m35.297187149s) [noise=true , address=145.79.227.52:4446 , piece_id=PV4IMXP5SFPR4HOK4K4FB7NK6DBNT7S6DWKBG65H5HHY3KVXA4DA, seg_pos={0 0}, upload=711423]: Uploading piece... Jan 8 12:49:37.080812517 (813h20m36.951895562s) [upload=711423]: Next returned nil segment Jan 8 12:49:37.080827575 (813h20m36.95191063s) [upload=711423]: Waiting for error group managing segments... Jan 8 12:49:37.410390734 (813h20m37.281473769s) [noise=true , address=145.79.227.30:4762 , piece_id=4627M3HYYIXI4SUKN2Z7HJ6UXOCDIGT63VVWLOCKRM23Q7WNB4OQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.417430659 (813h20m37.288513694s) [noise=true , address=145.79.227.50:4373 , piece_id=I3JJJBTRRC22BWKWR6KQIVDBEUF7HGJAHI2VHPRHOPQ5SSW2DGZQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.418274345 (813h20m37.28935738s) [noise=true , address=145.79.227.41:4087 , piece_id=DLEOLC6JZR72IPZ5BXHB3DPW55RNA7YFBQKVVVDMD2GR63NELK6Q, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.422770557 (813h20m37.293853592s) [noise=true , address=145.79.227.29:4943 , piece_id=5SLUJ5WZ5DKOWBFUBXGSGCGWSVXXQDCGPTZB5K2LAM4TJLELBMSQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.423585409 (813h20m37.294668454s) [noise=true , address=145.79.227.17:4972 , piece_id=HM5IX3CSC3YBEB65W2IQLYZALHKUMOGDYFTXG3GBJNUGBNHYQY2A, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.431555936 (813h20m37.302638971s) [noise=true , address=145.79.227.48:4303 , piece_id=L3KCG3IXMUYNW3DYUQAPSJZMEIXO34GLY4CXVPYGCFWKWBJDFNZQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.436586515 (813h20m37.30766955s) [noise=true , address=145.79.227.23:4546 , piece_id=NSSD6MR56ZCGB35XZWWVDJ3YLUUIMUSHZW643B2L2KQHYT7UVGSQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.438960084 (813h20m37.310043119s) [noise=true , address=145.79.227.12:4765 , piece_id=N3RV4OLEYXGDSBBZIGVUTHKBBTJPQ25OM5KS3QL2IGTJNKAAM3QQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.440418922 (813h20m37.311501967s) [noise=true , address=145.79.227.52:4446 , piece_id=PV4IMXP5SFPR4HOK4K4FB7NK6DBNT7S6DWKBG65H5HHY3KVXA4DA, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.498551524 (813h20m37.369634569s) [noise=true , address=145.79.227.39:4014 , piece_id=BRJSW66BJGLHP3YNWYQBFWNOXEO5DWUALF73GU6TXVXTBPTLFSOQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.695162104 (813h20m37.566245139s) [noise=true , address=77.90.31.18:4808 , piece_id=YYZP2EWLHVLPLBOWNP6V3WGMSTPQSKN6IMYSRSSHZ2QWKAV4C2ZA, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.699127946 (813h20m37.570210991s) [noise=true , address=77.90.31.14:4663 , piece_id=RKVXNWGN4633WKAWS2337N3OVBFXMHT4RP2VPGDOMRHDKXYLA3MA, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.729448060 (813h20m37.600531105s) [noise=true , address=77.90.31.15:4697 , piece_id=XFBYEWBTPC5MHBCVDOB7WRJL4CTBHTCJUIVCV3ZMBBLJWZFEDVGQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: <nil> Jan 8 12:49:37.729473708 (813h20m37.600556753s) [seg_pos={0 0}, upload=711423]: Segment reached optimal threshold of 13 pieces. Jan 8 12:49:37.729685752 (813h20m37.600768797s) [noise=true , address=45.91.116.11:4119 , piece_id=U5LLYWUM5CDOB6522ZMEMXWYEP67XWNBWY3MTOUX4L4X3PKSMSUQ, seg_pos={0 0}, upload=711423]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Jan 8 12:49:37.729694859 (813h20m37.600777905s) [noise=true , address=45.91.116.11:4119 , piece_id=U5LLYWUM5CDOB6522ZMEMXWYEP67XWNBWY3MTOUX4L4X3PKSMSUQ, seg_pos={0 0}, upload=711423]: Long tail cancellation due to optimal threshold Jan 8 12:49:37.729723093 (813h20m37.600806138s) [noise=true , address=77.90.31.11:4553 , piece_id=YIJFVBDYRTRWBWLFS5BS23Q7FJ5P5SSVU4IO2E2OK3X5GGE77WEA, seg_pos={0 0}, upload=711423]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Jan 8 12:49:37.729732351 (813h20m37.600815386s) [noise=true , address=77.90.31.11:4553 , piece_id=YIJFVBDYRTRWBWLFS5BS23Q7FJ5P5SSVU4IO2E2OK3X5GGE77WEA, seg_pos={0 0}, upload=711423]: Long tail cancellation due to optimal threshold Jan 8 12:49:37.729995942 (813h20m37.601078987s) [noise=true , address=45.91.116.6:4974 , piece_id=NDHADQKHSV2DTNQCMS563HAQBYYSESU4RAQMCW3PJRG6HJZYQQZA, seg_pos={0 0}, upload=711423]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: write tcp 188.166.225.170:35580->45.91.116.6:4974: use of closed network connection; write tcp 188.166.225.170:35580->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35580->45.91.116.6:4974: use of closed network connection Jan 8 12:49:37.730004018 (813h20m37.601087063s) [noise=true , address=45.91.116.6:4974 , piece_id=NDHADQKHSV2DTNQCMS563HAQBYYSESU4RAQMCW3PJRG6HJZYQQZA, seg_pos={0 0}, upload=711423]: Long tail cancellation due to optimal threshold Jan 8 12:49:37.730023204 (813h20m37.601106239s) [seg_pos={0 0}, upload=711423]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:49:37.730128605 (813h20m37.60121165s) [upload=711423]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:49:37.730134907 (813h20m37.601217952s) [upload=711423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:49:38.185849385 (813h20m38.05693245s) [upload=711423]: Done uploading segments... Jan 8 12:49:38.185872288 (813h20m38.056955323s) [upload=711423]: Upload finished. err: <nil> Duration: 30.004191861s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 53 out of 60 is failed (threshold=6) [.XXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.X.XXXX.XXX.XXXX] duration: 5.001403438s trace: https://jaeger.dev.storj.io/trace/34dc86ffa582a118 FAIL (2026-01-08 12:50:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001491446s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................X..X........X......X.X..............X....] duration: 25.737500317s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:50:02 UTC) orbiter warning: context deadline exceeded Duration: 25.737536616s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X........XX................X.........X..........X.X........] duration: 28.013477475s trace: https://jaeger.dev.storj.io/trace/71400ddaa07d17fe FAIL (2026-01-08 12:49:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: EH3VCU5F6BNWHJM7XFPF6CTXCHYWAAKILXLQ3F3IUBQBDTBTWEKA) context deadline exceeded; piecestore: (Node ID: 1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, Piece ID: 3DIFOHLVYFGPFGL6LJZ3EGIZBVJMKWZIFDXSB7XUASOPDKTA4KWA) context deadline exceeded ======LOGS====== (813h13m35.524219912s) [upload=711322]: Done starting upload Jan 8 12:42:35.653228631 (813h13m35.524311666s) [upload=711322]: Uploading segments... Jan 8 12:42:35.653328681 (813h13m35.524411736s) [upload=711322]: Got next segment. Inline: false Jan 8 12:42:35.653370711 (813h13m35.524453756s) [upload=711322]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 8 12:42:35.653376201 (813h13m35.524459236s) [upload=711322]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 8 12:42:35.898540144 (813h13m35.769623189s) [seg_pos={0 0}, upload=711322]: Begin upload segment... Jan 8 12:42:35.898588416 (813h13m35.769671451s) [seg_pos={0 0}, upload=711322]: Done begin upload segment. Jan 8 12:42:35.898617030 (813h13m35.769700065s) [noise=true , address=77.90.31.19:4841 , piece_id=EH3VCU5F6BNWHJM7XFPF6CTXCHYWAAKILXLQ3F3IUBQBDTBTWEKA, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.898777216 (813h13m35.769860251s) [noise=true , address=45.91.116.6:4974 , piece_id=ZGJDC5DJ35FUKINUUBWB3BB3TE37V55HUA62UQVFZMJRDV2J574Q, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.899214167 (813h13m35.770297212s) [noise=true , address=145.79.227.44:4193 , piece_id=R446EZ2B3THR2F3CU4G5VYXFN4HWFSN5NTRUF5ZFFOZPT4EZMEAQ, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.899501955 (813h13m35.77058499s) [noise=true , address=145.79.227.29:4942 , piece_id=FOTNTVFHKEG22P2KH5SA325VHWXZZHDXAKYWAWBGJF7JBMSKO32Q, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.899771348 (813h13m35.770854383s) [noise=true , address=45.91.116.10:4084 , piece_id=PUMPJVE3CEIDIIS3DYNRCERDVKHZOTKADXVXPC4VPUTCYH2EFFKA, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.900047234 (813h13m35.771130269s) [noise=true , address=45.91.116.8:4045 , piece_id=EUUSC2XSVSPHJNWJBEGXQQD3MAC6SI2ZOLDTJKZPOGPL3XBJK2RA, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.900319973 (813h13m35.771403008s) [noise=true , address=45.91.116.11:4120 , piece_id=PYBYSVRS66NFRC3EMSYGERK6YTIQYPU6PY74ZGK2TLUGUUCWQR6A, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.900569308 (813h13m35.771652343s) [noise=true , address=145.79.227.18:4007 , piece_id=X6PVJ3A45SS5GQN7RUKIXPGRHXEGZAT45QP3JQ4VITKJSUYSCJGA, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.900902823 (813h13m35.771985868s) [noise=true , address=45.91.116.7:4010 , piece_id=NR6L3M54M6MNLI5DV7BL5I2YWYMUJP2FQT76QNPJZTNIWWETHWQA, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.902363944 (813h13m35.773446979s) [noise=true , address=145.79.227.37:4566 , piece_id=FDECEFZTO4YQUCARTGCD2FHBBSWA64BIHVB62TOEORYI66BSI7YQ, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.902796709 (813h13m35.773879744s) [noise=true , address=45.91.116.12:4228 , piece_id=WB7ZUONP45E4ZTKF3B7OG7KHCX5XI3VYASZPIDTTIYZU37RS5BTA, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.909685846 (813h13m35.780768881s) [noise=true , address=145.79.227.17:4971 , piece_id=Y5LEEIFD73HLTVUQ3QSHBXBEZZTQG2QNWFOD2IAMBFPGGUGYRAMQ, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.911717616 (813h13m35.782800661s) [noise=true , address=45.91.116.14:4190 , piece_id=3DIFOHLVYFGPFGL6LJZ3EGIZBVJMKWZIFDXSB7XUASOPDKTA4KWA, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.912830555 (813h13m35.78391359s) [noise=true , address=145.79.227.46:4603 , piece_id=3IDZQEIK3CVRYJHCXBXAHBIN6SNSPSRGSECOMNH7VE2GSN6MIZYQ, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.915309455 (813h13m35.78639249s) [noise=true , address=145.79.227.21:4474 , piece_id=ZQY5DAUQAASTDMXAZQR2TBOJ57RBAJLVRXMFIFKY747VBLS6WR5Q, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:35.916592988 (813h13m35.787676023s) [noise=true , address=145.79.227.30:4762 , piece_id=5MFSHCDCJVETFYCEQ4NIIZOIRRXOUOONN3T2QOAPKJHBOSVFFP2A, seg_pos={0 0}, upload=711322]: Uploading piece... Jan 8 12:42:37.499691456 (813h13m37.370774491s) [upload=711322]: Next returned nil segment Jan 8 12:42:37.499709089 (813h13m37.370792134s) [upload=711322]: Waiting for error group managing segments... Jan 8 12:42:37.879628492 (813h13m37.750711527s) [noise=true , address=145.79.227.29:4942 , piece_id=FOTNTVFHKEG22P2KH5SA325VHWXZZHDXAKYWAWBGJF7JBMSKO32Q, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:37.880370436 (813h13m37.751453501s) [noise=true , address=145.79.227.44:4193 , piece_id=R446EZ2B3THR2F3CU4G5VYXFN4HWFSN5NTRUF5ZFFOZPT4EZMEAQ, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:37.883356811 (813h13m37.754439846s) [noise=true , address=145.79.227.18:4007 , piece_id=X6PVJ3A45SS5GQN7RUKIXPGRHXEGZAT45QP3JQ4VITKJSUYSCJGA, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:37.905875360 (813h13m37.776958395s) [noise=true , address=145.79.227.37:4566 , piece_id=FDECEFZTO4YQUCARTGCD2FHBBSWA64BIHVB62TOEORYI66BSI7YQ, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:37.908869130 (813h13m37.779952165s) [noise=true , address=145.79.227.17:4971 , piece_id=Y5LEEIFD73HLTVUQ3QSHBXBEZZTQG2QNWFOD2IAMBFPGGUGYRAMQ, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:37.915399445 (813h13m37.78648248s) [noise=true , address=145.79.227.30:4762 , piece_id=5MFSHCDCJVETFYCEQ4NIIZOIRRXOUOONN3T2QOAPKJHBOSVFFP2A, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:37.919452292 (813h13m37.790535327s) [noise=true , address=145.79.227.46:4603 , piece_id=3IDZQEIK3CVRYJHCXBXAHBIN6SNSPSRGSECOMNH7VE2GSN6MIZYQ, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:37.928963723 (813h13m37.800046758s) [noise=true , address=145.79.227.21:4474 , piece_id=ZQY5DAUQAASTDMXAZQR2TBOJ57RBAJLVRXMFIFKY747VBLS6WR5Q, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:38.218673346 (813h13m38.089756382s) [noise=true , address=77.90.31.19:4841 , piece_id=EH3VCU5F6BNWHJM7XFPF6CTXCHYWAAKILXLQ3F3IUBQBDTBTWEKA, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:38.320578907 (813h13m38.191661952s) [noise=true , address=45.91.116.6:4974 , piece_id=ZGJDC5DJ35FUKINUUBWB3BB3TE37V55HUA62UQVFZMJRDV2J574Q, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:38.330651917 (813h13m38.201734972s) [noise=true , address=45.91.116.8:4045 , piece_id=EUUSC2XSVSPHJNWJBEGXQQD3MAC6SI2ZOLDTJKZPOGPL3XBJK2RA, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:38.356860992 (813h13m38.227944027s) [noise=true , address=45.91.116.7:4010 , piece_id=NR6L3M54M6MNLI5DV7BL5I2YWYMUJP2FQT76QNPJZTNIWWETHWQA, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:38.420431920 (813h13m38.291514955s) [noise=true , address=45.91.116.14:4190 , piece_id=3DIFOHLVYFGPFGL6LJZ3EGIZBVJMKWZIFDXSB7XUASOPDKTA4KWA, seg_pos={0 0}, upload=711322]: Done uploading piece. err: <nil> Jan 8 12:42:38.420460655 (813h13m38.2915437s) [seg_pos={0 0}, upload=711322]: Segment reached optimal threshold of 13 pieces. Jan 8 12:42:38.420648041 (813h13m38.291731076s) [noise=true , address=45.91.116.11:4120 , piece_id=PYBYSVRS66NFRC3EMSYGERK6YTIQYPU6PY74ZGK2TLUGUUCWQR6A, seg_pos={0 0}, upload=711322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y9H9KNVuhzdB1XAPiwzHsFWYR4SrgB8Eff9HY6q6jmsYMTst8): protocol: write tcp 188.166.225.170:42392->45.91.116.11:4120: use of closed network connection; write tcp 188.166.225.170:42392->45.91.116.11:4120: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42392->45.91.116.11:4120: use of closed network connection Jan 8 12:42:38.420656257 (813h13m38.291739292s) [noise=true , address=45.91.116.11:4120 , piece_id=PYBYSVRS66NFRC3EMSYGERK6YTIQYPU6PY74ZGK2TLUGUUCWQR6A, seg_pos={0 0}, upload=711322]: Long tail cancellation due to optimal threshold Jan 8 12:42:38.420775294 (813h13m38.291858329s) [noise=true , address=45.91.116.10:4084 , piece_id=PUMPJVE3CEIDIIS3DYNRCERDVKHZOTKADXVXPC4VPUTCYH2EFFKA, seg_pos={0 0}, upload=711322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj): protocol: expected piece hash; context canceled; EOF Jan 8 12:42:38.420781495 (813h13m38.29186453s) [noise=true , address=45.91.116.10:4084 , piece_id=PUMPJVE3CEIDIIS3DYNRCERDVKHZOTKADXVXPC4VPUTCYH2EFFKA, seg_pos={0 0}, upload=711322]: Long tail cancellation due to optimal threshold Jan 8 12:42:38.421127114 (813h13m38.292210169s) [noise=true , address=45.91.116.12:4228 , piece_id=WB7ZUONP45E4ZTKF3B7OG7KHCX5XI3VYASZPIDTTIYZU37RS5BTA, seg_pos={0 0}, upload=711322]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qsv8fYepBmZ5RYqo42q94CxRqUUmW2xP5abmXwM5GukQRXfPA): protocol: read tcp 188.166.225.170:41622->45.91.116.12:4228: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 8 12:42:38.421145979 (813h13m38.292229024s) [noise=true , address=45.91.116.12:4228 , piece_id=WB7ZUONP45E4ZTKF3B7OG7KHCX5XI3VYASZPIDTTIYZU37RS5BTA, seg_pos={0 0}, upload=711322]: Long tail cancellation due to optimal threshold Jan 8 12:42:38.421189361 (813h13m38.292272396s) [seg_pos={0 0}, upload=711322]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 8 12:42:38.421351711 (813h13m38.292434756s) [upload=711322]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 8 12:42:38.421360728 (813h13m38.292443763s) [upload=711322]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 8 12:42:38.712995968 (813h13m38.584079023s) [upload=711322]: Done uploading segments... Jan 8 12:42:38.713017880 (813h13m38.584100915s) [upload=711322]: Upload finished. err: <nil> Duration: 28.013507552s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................X..X........X......X.X..............X...] duration: 25.343751355s trace: https://jaeger.dev.storj.io/trace/19b8acde79c35373 FAIL (2026-01-08 12:49:04 UTC) orbiter warning: context deadline exceeded Duration: 25.343773427s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................X.......X.........................] duration: 30.646916134s trace: https://jaeger.dev.storj.io/trace/f8dd51e1a9bb81f FAIL (2026-01-08 12:49:03 UTC) orbiter warning: context deadline exceeded Duration: 30.646933557s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)