Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXX] duration:
5.001479989s trace:
https://jaeger.dev.storj.io/trace/45538311f861cfc7
FAIL (2026-01-12 09:24:31 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001503502s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
261.108386ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:24:23 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 261.131701ms
edge
linksharing
bgp
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [X.....XX.X...................................X...........X..] duration:
19.310909698s trace:
https://jaeger.dev.storj.io/trace/3e3313cc22a76e96
FAIL (2026-01-12 09:24:09 UTC)
orbiter warning: context deadline exceeded
Duration: 19.310931979s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
238.891302ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2220c356c01a97b6
FAIL (2026-01-12 09:23:59 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded
Duration: 238.992846ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
252.880127ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:23:56 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 252.908322ms
edge
linksharing
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................................XX........] duration:
5.719301816s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:23:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 5.719331542s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXX] duration:
5.001746661s trace:
https://jaeger.dev.storj.io/trace/53b71cbc8af4228d
FAIL (2026-01-12 09:23:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001774145s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
252.021333ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:23:32 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 252.046382ms
edge
linksharing
bgp
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X.....XX.X...................................X...........X.] duration:
19.787956992s trace:
https://jaeger.dev.storj.io/trace/3e3313cc22a76e96
FAIL (2026-01-12 09:23:30 UTC)
orbiter warning: context deadline exceeded
Duration: 19.787971409s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
250.255472ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2220c356c01a97b6
FAIL (2026-01-12 09:23:23 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded
Duration: 250.285309ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXX] duration:
5.001660785s trace:
https://jaeger.dev.storj.io/trace/7d36408f90f39211
FAIL (2026-01-12 09:23:18 UTC)
orbiter warning: context deadline exceeded
Duration: 5.00168939s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X.....XX.X...................................X...........X] duration:
30.001282119s trace:
https://jaeger.dev.storj.io/trace/3e3313cc22a76e96
FAIL (2026-01-12 09:23:10 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001305864s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
247.646783ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:23:06 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 247.673706ms
edge
linksharing
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................................XX.......] duration:
5.728771344s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:22:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 5.728796311s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
239.064881ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:22:47 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 239.091152ms
edge
linksharing
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXX] duration:
5.001419832s trace:
https://jaeger.dev.storj.io/trace/ec7d6599bbc9755
FAIL (2026-01-12 09:22:42 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001509092s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
250.322008ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2220c356c01a97b6
FAIL (2026-01-12 09:22:27 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded
Duration: 250.347134ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXX] duration:
5.001289419s trace:
https://jaeger.dev.storj.io/trace/7afdbdadc1cbab8e
FAIL (2026-01-12 09:22:23 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001313945s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................................XX......] duration:
5.763793539s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:22:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 5.763823936s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [...X.....XX.X...................................X...........] duration:
18.376189063s trace:
https://jaeger.dev.storj.io/trace/6e4e88c8ac4918c3
FAIL (2026-01-12 09:21:48 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.376206968s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
244.017704ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:21:46 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 244.049403ms
edge
linksharing
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXX] duration:
5.001267811s trace:
https://jaeger.dev.storj.io/trace/1c7022c1ea9fe22d
FAIL (2026-01-12 09:21:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001294843s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [....X.....XX.X...................................X..........] duration:
21.547717799s trace:
https://jaeger.dev.storj.io/trace/6e4e88c8ac4918c3
FAIL (2026-01-12 09:21:29 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.547746875s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
255.580027ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2220c356c01a97b6
FAIL (2026-01-12 09:21:22 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded
Duration: 255.607588ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration:
5.715197169s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:21:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 5.715215975s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXXX] duration:
5.002315413s trace:
https://jaeger.dev.storj.io/trace/6bc6f8fe78a23b80
FAIL (2026-01-12 09:21:19 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.002337415s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....X.....XX.X...................................X.........] duration:
21.210420409s trace:
https://jaeger.dev.storj.io/trace/6e4e88c8ac4918c3
FAIL (2026-01-12 09:20:55 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.210449404s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
256.056542ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:20:47 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 256.083604ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
251.953851ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2220c356c01a97b6
FAIL (2026-01-12 09:20:31 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded
Duration: 251.983318ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXXX] duration:
5.00152419s trace:
https://jaeger.dev.storj.io/trace/6d9416983385afa1
FAIL (2026-01-12 09:20:31 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001588031s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [......X.....XX.X...................................X........] duration:
18.062757952s trace:
https://jaeger.dev.storj.io/trace/6e4e88c8ac4918c3
FAIL (2026-01-12 09:20:23 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.062782358s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................................XX....] duration:
5.805551156s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:20:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 5.805580083s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
243.674505ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:20:11 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 243.687923ms
edge
linksharing
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXXX] duration:
5.000472126s trace:
https://jaeger.dev.storj.io/trace/56a8272115250ce2
FAIL (2026-01-12 09:19:38 UTC)
orbiter warning: context deadline exceeded
Duration: 5.000493728s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......X.....XX.X...................................X.......] duration:
18.47421384s trace:
https://jaeger.dev.storj.io/trace/6e4e88c8ac4918c3
FAIL (2026-01-12 09:19:31 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.47426656s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..........X.................................................] duration:
248.47197ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:19:20 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 248.500013ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
251.03498ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2220c356c01a97b6
FAIL (2026-01-12 09:19:20 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded
Duration: 251.060338ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
5.90937131s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:19:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 5.909402771s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXXX] duration:
5.001199747s trace:
https://jaeger.dev.storj.io/trace/708742c06693ddd3
FAIL (2026-01-12 09:19:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001224926s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [........X.....XX.X...................................X......] duration:
22.009883931s trace:
https://jaeger.dev.storj.io/trace/6e4e88c8ac4918c3
FAIL (2026-01-12 09:18:48 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.009898249s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
5.70687062s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:18:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 5.706897001s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.........X.....XX.X...................................X.....] duration:
27.020317785s trace:
https://jaeger.dev.storj.io/trace/6e4e88c8ac4918c3
FAIL (2026-01-12 09:18:26 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 27.020350989s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
255.775026ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2220c356c01a97b6
FAIL (2026-01-12 09:18:22 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded
Duration: 255.803329ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXXX] duration:
5.001028629s trace:
https://jaeger.dev.storj.io/trace/7741d6f757b5f96d
FAIL (2026-01-12 09:18:19 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001049408s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...........X................................................] duration:
251.88017ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6909b032d2b9dbf8
FAIL (2026-01-12 09:18:06 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 251.916159ms
edge
linksharing
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
59 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXXXXXXXX] duration:
5.002054148s trace:
https://jaeger.dev.storj.io/trace/2d8c189d6b17ded9
FAIL (2026-01-12 09:17:54 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.002073143s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
5.752821662s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:17:33 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 5.752849636s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [..........X.....XX.X...................................X....] duration:
18.646256999s trace:
https://jaeger.dev.storj.io/trace/6e4e88c8ac4918c3
FAIL (2026-01-12 09:17:30 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.646321302s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
30.003843152s trace:
https://jaeger.dev.storj.io/trace/ab64659e76d444f
FAIL (2026-01-12 09:17:26 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, Piece ID: LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA) context deadline exceeded; piecestore: (Node ID: 19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, Piece ID: MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A) context deadline exceeded; piecestore: (Node ID: 1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8, Piece ID: WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA) context deadline exceeded; piecestore: (Node ID: 125GxLsL5UtXkuJ1LFvF42N9iZkKtRK3rq63xdR25JQbtwoT1L5, Piece ID: ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ) context deadline exceeded; piecestore: (Node ID: 12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj, Piece ID: AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ) context deadline exceeded; piecestore: (Node ID: 12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, Piece ID: CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA) context deadline exceeded
======LOGS======
Jan 12 09:16:56.792600762 (905h47m56.663683807s) [upload=792826]: Starting upload
Jan 12 09:16:56.792703167 (905h47m56.663786212s) [upload=792826]: Done starting upload
Jan 12 09:16:56.792791775 (905h47m56.66387485s) [upload=792826]: Uploading segments...
Jan 12 09:16:56.792986346 (905h47m56.664069391s) [upload=792826]: Got next segment. Inline: false
Jan 12 09:16:56.793042272 (905h47m56.664125317s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 12 09:16:56.793048945 (905h47m56.66413198s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 12 09:16:57.303000354 (905h47m57.174083399s) [seg_pos={0 0}, upload=792826]: Begin upload segment...
Jan 12 09:16:57.303050018 (905h47m57.174133063s) [seg_pos={0 0}, upload=792826]: Done begin upload segment.
Jan 12 09:16:57.303099402 (905h47m57.174182438s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303125853 (905h47m57.174208888s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.303863027 (905h47m57.174946062s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.304692776 (905h47m57.175775821s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305125821 (905h47m57.176208856s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.305559506 (905h47m57.176642541s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.306039110 (905h47m57.177122145s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307511213 (905h47m57.178594248s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.307674754 (905h47m57.178757789s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309207323 (905h47m57.180290368s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.309324225 (905h47m57.18040727s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.310985969 (905h47m57.182069004s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.311179898 (905h47m57.182262943s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.313122637 (905h47m57.184205683s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.315131393 (905h47m57.186214438s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:57.319394119 (905h47m57.190477154s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Uploading piece...
Jan 12 09:16:59.087297100 (905h47m58.958380135s) [upload=792826]: Next returned nil segment
Jan 12 09:16:59.087310024 (905h47m58.958393059s) [upload=792826]: Waiting for error group managing segments...
Jan 12 09:16:59.429705807 (905h47m59.300788842s) [noise=true , address=77.90.31.11:4553 , piece_id=YUHJ6FJACICRGDJ6RPZQVADCT6UTLCLDHIJZZ7MTGJOXAJLEAQHA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.448043334 (905h47m59.319126379s) [noise=true , address=77.90.31.14:4663 , piece_id=Q7QJLRLXVHJOBIUWVX6A7APFL2UE2QKXRQQCT767GGS5QDR4P75Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.456028489 (905h47m59.327111524s) [noise=true , address=145.79.227.5:4223 , piece_id=NT4FP5F3Z7VI5BLGRSLLRVCGRCEE2XFNZONKECRPOGITVAW5HYOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.486510770 (905h47m59.357593806s) [noise=true , address=145.79.227.43:4157 , piece_id=2QI5I6K5V6ZWQYFCCKNJOMDFTFXH65TC44BWPLYXWAWQHJQHHJ7A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.494721013 (905h47m59.365804058s) [noise=true , address=145.79.227.44:4193 , piece_id=WLUJ7242QP2MJ5REPRCYK3LIKIWLBUH2JVUX6QXGIAPW4IKFANAQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.542756890 (905h47m59.413839965s) [noise=true , address=145.79.227.15:4873 , piece_id=DGDGA54UVGUHV2P6ILI5FTDQX2VG5XMERS74XJADER3VCXV6OGEA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.555273382 (905h47m59.426356417s) [noise=true , address=145.79.227.46:4601 , piece_id=ILQV7LYBEVQL2IG3SDDH2BUJ5JIXNDEMPT2JLYJNVLHT64IBZEKQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.619530646 (905h47m59.490613681s) [noise=true , address=145.79.227.45:4231 , piece_id=OJ3KP5XXUTXSI5IG7BMVIF46E5QXVLCVFYYLPWYANYZNLWG5NGOA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.797964675 (905h47m59.669047721s) [noise=true , address=145.79.227.31:4799 , piece_id=ECBF2XWHBWEOERAIOAQAI6RUY32DDQLLKLDPKWZX4E537K7X7K5Q, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.808633430 (905h47m59.679716475s) [noise=true , address=145.79.227.51:4409 , piece_id=AX7DTBGHU2HAKOZ4FAH3XKYQPMK4L3N7MTQFREPIMVGGROSUF2FQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:16:59.943655669 (905h47m59.814738714s) [noise=true , address=145.79.227.23:4547 , piece_id=LVXZYBX2D6JDRQQN6JPVZHNQIVP4G5BW3TCETP23W22MHCEZOQKA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175177895 (905h48m0.046260931s) [noise=true , address=145.79.227.18:4007 , piece_id=MTRREB3KN6EXE3SAIFZ5HBHPQB7LEQQSRUVI7ANOBCTMUEH7X36A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175884611 (905h48m0.046967656s) [noise=true , address=145.79.227.33:4871 , piece_id=CUYB4D5YJ7KNDB4DC3YAF53PTYXQO2UWZPFUYQLKPYHZZJZ5D4KQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: <nil>
Jan 12 09:17:00.175909178 (905h48m0.046992223s) [seg_pos={0 0}, upload=792826]: Segment reached optimal threshold of 13 pieces.
Jan 12 09:17:00.176276938 (905h48m0.047359973s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55404->145.79.227.17:4970: use of closed network connection
Jan 12 09:17:00.176286415 (905h48m0.04736945s) [noise=true , address=145.79.227.17:4970 , piece_id=B23BICB6N7NQJJ3JA7KCMDOUVIB36XFQRUIBFEZN3Z7BHP2LAL2A, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176336792 (905h48m0.047419847s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176347923 (905h48m0.047430968s) [noise=true , address=145.79.227.21:4474 , piece_id=5WNFIMZC2IBSLSYYB3A3KYEIRW32CZGIV2SVR7TNNQMJAKULPLUA, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176508509 (905h48m0.047591554s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125pLUE8mHgK5ZuhphNhDfjNB2kTM88jKNtraf7MMUDC9DU2YVW): protocol: expected piece hash; context canceled; EOF
Jan 12 09:17:00.176519770 (905h48m0.047602805s) [noise=true , address=145.79.227.8:4330 , piece_id=I6DKLETV33CG3E77SYDQVUPNI5FYG3NQNBIHWV3TCHH6BLBDGOSQ, seg_pos={0 0}, upload=792826]: Long tail cancellation due to optimal threshold
Jan 12 09:17:00.176538286 (905h48m0.047621321s) [seg_pos={0 0}, upload=792826]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan 12 09:17:00.176698160 (905h48m0.047781195s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 12 09:17:00.176704502 (905h48m0.047787537s) [upload=792826]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 12 09:17:00.493070711 (905h48m0.364153756s) [upload=792826]: Done uploading segments...
Jan 12 09:17:00.493089967 (905h48m0.364173002s) [upload=792826]: Upload finished. err: <nil>
Duration: 30.003854915s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
261.690199ms remote:
"tcp:207.211.208.129:443" target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2220c356c01a97b6
FAIL (2026-01-12 09:17:13 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded
Duration: 261.716519ms
edge
gateway
vhost
bgp
eu1
|