Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 18.785703583s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:40:20 UTC) orbiter warning: context deadline exceeded Duration: 18.78572795s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.00133388s trace: https://jaeger.dev.storj.io/trace/69544cad075a47c5 FAIL (2026-01-12 09:40:17 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001482803s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 15.512507841s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:39:55 UTC) orbiter warning: context deadline exceeded Duration: 15.512554068s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.000959374s trace: https://jaeger.dev.storj.io/trace/14ee8b6639ed5ba5 FAIL (2026-01-12 09:39:54 UTC) orbiter warning: context deadline exceeded Duration: 5.00097364s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................X...........X...X....X...........] duration: 25.650674716s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:39:44 UTC) orbiter warning: context deadline exceeded Duration: 25.650702159s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 5.821070082s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:39:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.821097685s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 20.767141347s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:39:26 UTC) orbiter warning: context deadline exceeded Duration: 20.767169149s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.000528512s trace: https://jaeger.dev.storj.io/trace/3e556490d85ed41e FAIL (2026-01-12 09:39:25 UTC) orbiter warning: context deadline exceeded Duration: 5.000550575s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................X...........X...X....X..........] duration: 18.826177383s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:39:00 UTC) orbiter warning: context deadline exceeded Duration: 18.826200528s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.002638204s trace: https://jaeger.dev.storj.io/trace/323451589fde1f83 FAIL (2026-01-12 09:38:54 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002678461s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 5.788034786s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:38:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.788058331s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 19.931346012s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:38:33 UTC) orbiter warning: context deadline exceeded Duration: 19.931373635s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................X...........X...X....X.........] duration: 29.485829263s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:38:20 UTC) orbiter warning: context deadline exceeded Duration: 29.485855093s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 31.018082138s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:38:10 UTC) orbiter warning: context deadline exceeded Duration: 31.018111965s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.00116659s trace: https://jaeger.dev.storj.io/trace/4f3acdfc6568d438 FAIL (2026-01-12 09:37:46 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001211555s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 5.758164083s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:37:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.758195642s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 18.511746246s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:37:25 UTC) orbiter warning: context deadline exceeded Duration: 18.511762447s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 5.745044649s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:37:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.745071492s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 18.486931458s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:37:06 UTC) orbiter warning: context deadline exceeded Duration: 18.48695412s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................X...........X...X....X........] duration: 17.050341221s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:37:05 UTC) orbiter warning: context deadline exceeded Duration: 17.050905255s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001725725s trace: https://jaeger.dev.storj.io/trace/768d9441d00758bf FAIL (2026-01-12 09:36:35 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001747798s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 5.717379895s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:36:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.717404052s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 28.061720573s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:36:15 UTC) orbiter warning: context deadline exceeded Duration: 28.061746923s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................X...........X...X....X.......] duration: 24.267310147s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:36:06 UTC) orbiter warning: context deadline exceeded Duration: 24.267321488s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................X...........X...X....X......] duration: 26.158447657s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:35:41 UTC) orbiter warning: context deadline exceeded Duration: 26.158471122s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.00107013s trace: https://jaeger.dev.storj.io/trace/35255bbd2a0f09d8 FAIL (2026-01-12 09:35:37 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001091411s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 5.72098178s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:35:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.721088463s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 17.843539281s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:34:59 UTC) orbiter warning: context deadline exceeded Duration: 17.843566481s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001612711s trace: https://jaeger.dev.storj.io/trace/4cf175591c1988c6 FAIL (2026-01-12 09:34:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001636947s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................X...........X...X....X.....] duration: 19.162581748s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:34:24 UTC) orbiter warning: context deadline exceeded Duration: 19.162645278s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001374127s trace: https://jaeger.dev.storj.io/trace/18f9ebeafd27e637 FAIL (2026-01-12 09:34:21 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001396861s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 27.816571192s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:34:12 UTC) orbiter warning: context deadline exceeded Duration: 27.816583635s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 5.75522723s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:34:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.755250024s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001194365s trace: https://jaeger.dev.storj.io/trace/1a2aa205e005182f FAIL (2026-01-12 09:33:54 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001214193s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 19.245998026s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:33:44 UTC) orbiter warning: context deadline exceeded Duration: 19.246022712s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................X...........X...X....X....] duration: 19.047800501s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:33:36 UTC) orbiter warning: context deadline exceeded Duration: 19.047827493s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.001437687s trace: https://jaeger.dev.storj.io/trace/6b7416ba98ae7a21 FAIL (2026-01-12 09:33:27 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001460641s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 5.744935153s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:33:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.744962376s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 8.279794346s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:33:07 UTC) orbiter warning: context deadline exceeded Duration: 8.279820136s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.002143503s trace: https://jaeger.dev.storj.io/trace/23a8ddf6098815d4 FAIL (2026-01-12 09:33:05 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002166257s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................X...........X...X....X...] duration: 20.586478321s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:33:03 UTC) orbiter warning: context deadline exceeded Duration: 20.586503751s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 17.280568163s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:32:53 UTC) orbiter warning: context deadline exceeded Duration: 17.28058134s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 21.147535597s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:32:36 UTC) orbiter warning: context deadline exceeded Duration: 21.147563481s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 5.790481544s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:32:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.790503775s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.00174886s trace: https://jaeger.dev.storj.io/trace/9d2481f79ee78ec FAIL (2026-01-12 09:32:18 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001765601s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 5.801843869s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:31:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.80187051s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X...................................X...........X...X....X..] duration: 21.074640004s trace: https://jaeger.dev.storj.io/trace/70187648ad205ded FAIL (2026-01-12 09:31:55 UTC) orbiter warning: context deadline exceeded Duration: 21.074667757s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 5.000847997s trace: https://jaeger.dev.storj.io/trace/19be2bdbf75203de FAIL (2026-01-12 09:31:24 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000863787s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 23.856839507s trace: https://jaeger.dev.storj.io/trace/9d61d7af563438b FAIL (2026-01-12 09:31:24 UTC) orbiter warning: context deadline exceeded Duration: 23.856874623s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 5.749365983s trace: https://jaeger.dev.storj.io/trace/ab64659e76d444f FAIL (2026-01-12 09:31:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded ======LOGS====== Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments... Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment... Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment. Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece... Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments... Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil> Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces. Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments... Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil> Duration: 5.74939546s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)