Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001138323s trace:
https://jaeger.dev.storj.io/trace/1ccca370716d7bcd
FAIL (2026-01-16 13:37:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001159805s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001501755s trace:
https://jaeger.dev.storj.io/trace/4c9f2b7ff6f4cc17
FAIL (2026-01-16 13:37:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001526562s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [...XX.....X..XX..X.............X.X............X.............] duration:
22.170047504s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:37:25 UTC)
orbiter warning: context deadline exceeded
Duration: 22.170074404s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
49 out of 60 is failed (threshold=3) [X..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXX] duration:
30.001333494s trace:
https://jaeger.dev.storj.io/trace/53379837b05189ec
FAIL (2026-01-16 13:37:13 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001359274s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
17.106713617s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:37:04 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 17.106889812s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [....XX.....X..XX..X.............X.X............X............] duration:
37.520871141s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:36:48 UTC)
orbiter warning: context deadline exceeded
Duration: 37.520900908s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000797495s trace:
https://jaeger.dev.storj.io/trace/23005079047b1e83
FAIL (2026-01-16 13:36:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00081564s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
17.930613798s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:36:25 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 17.930636491s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000506981s trace:
https://jaeger.dev.storj.io/trace/3be7ffb23c6f2345
FAIL (2026-01-16 13:36:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000518954s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X.............X........] duration:
4.758462715s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:36:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 4.758489909s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
49 out of 60 is failed (threshold=3) [XX..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXX] duration:
30.001869946s trace:
https://jaeger.dev.storj.io/trace/32a4148c3ee6d51e
FAIL (2026-01-16 13:36:04 UTC)
orbiter warning: context deadline exceeded
Duration: 30.00189323s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......................................X.............X.......] duration:
4.285375381s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:35:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 4.285404756s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [.....XX.....X..XX..X.............X.X............X...........] duration:
26.027102069s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:35:46 UTC)
orbiter warning: context deadline exceeded
Duration: 26.027195486s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X.............X......] duration:
4.727521422s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:35:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 4.72754759s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
49 out of 60 is failed (threshold=3) [XXX..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXX] duration:
30.001033604s trace:
https://jaeger.dev.storj.io/trace/6e5381d20a4912b
FAIL (2026-01-16 13:35:32 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001057429s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
23.980920074s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:35:32 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 23.980948379s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001501556s trace:
https://jaeger.dev.storj.io/trace/6bf833836944fa8b
FAIL (2026-01-16 13:35:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001521593s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000918776s trace:
https://jaeger.dev.storj.io/trace/28fb13208e22b686
FAIL (2026-01-16 13:35:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000943964s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [......XX.....X..XX..X.............X.X............X..........] duration:
26.226745942s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:35:12 UTC)
orbiter warning: context deadline exceeded
Duration: 26.226772442s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
48 out of 60 is failed (threshold=3) [.XXX..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXX] duration:
30.001345799s trace:
https://jaeger.dev.storj.io/trace/6d52f50c12f4c6fb
FAIL (2026-01-16 13:34:59 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001368292s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
13.919664931s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:34:37 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 13.919689267s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........................................X.............X.....] duration:
4.780883507s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:34:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 4.780904666s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000676394s trace:
https://jaeger.dev.storj.io/trace/2cda6a754373d9cd
FAIL (2026-01-16 13:34:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000696634s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [.......XX.....X..XX..X.............X.X............X.........] duration:
17.570380293s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:34:02 UTC)
orbiter warning: context deadline exceeded
Duration: 17.570408355s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
13.18081071s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:34:01 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 13.180834175s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
47 out of 60 is failed (threshold=3) [..XXX..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXX] duration:
30.001020028s trace:
https://jaeger.dev.storj.io/trace/5ce6337c5f4dea19
FAIL (2026-01-16 13:34:00 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001040997s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001442272s trace:
https://jaeger.dev.storj.io/trace/2158144276a3946e
FAIL (2026-01-16 13:33:34 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00145709s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [........XX.....X..XX..X.............X.X............X........] duration:
18.394768895s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:33:26 UTC)
orbiter warning: context deadline exceeded
Duration: 18.394797158s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X.............X....] duration:
4.42532839s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:33:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 4.425358026s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
47 out of 60 is failed (threshold=3) [X..XXX..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXX] duration:
30.001272238s trace:
https://jaeger.dev.storj.io/trace/2999c0f4649dfc4b
FAIL (2026-01-16 13:33:22 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001296784s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001150947s trace:
https://jaeger.dev.storj.io/trace/1e012983fc6c7964
FAIL (2026-01-16 13:33:03 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001161316s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
8.854599676s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:33:02 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 8.854628592s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
46 out of 60 is failed (threshold=3) [.X..XXX..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXX] duration:
30.001321944s trace:
https://jaeger.dev.storj.io/trace/785ba4ac51c788f6
FAIL (2026-01-16 13:32:35 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001331582s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
14.930335985s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:32:33 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 14.930360031s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........................................X.............X...] duration:
4.298163831s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:32:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 4.298204819s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [.........XX.....X..XX..X.............X.X............X.......] duration:
18.421610043s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:32:20 UTC)
orbiter warning: context deadline exceeded
Duration: 18.42164053s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
7.356853435s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:32:09 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 7.356884364s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
45 out of 60 is failed (threshold=3) [..X..XXX..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXX] duration:
30.001175012s trace:
https://jaeger.dev.storj.io/trace/42eb1c88a73ff321
FAIL (2026-01-16 13:32:05 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001195974s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........................................X.............X..] duration:
4.444049766s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:32:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 4.44407807s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000855585s trace:
https://jaeger.dev.storj.io/trace/16fa650ae57b507
FAIL (2026-01-16 13:32:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000877496s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
17.45528166s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:31:54 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 17.455307118s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [..........XX.....X..XX..X.............X.X............X......] duration:
27.832107269s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:31:41 UTC)
orbiter warning: context deadline exceeded
Duration: 27.832132447s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [............................................X.............X.] duration:
5.396387029s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:31:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 5.396404011s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001415782s trace:
https://jaeger.dev.storj.io/trace/220cacbd5b93626a
FAIL (2026-01-16 13:31:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001439146s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.............................................X.............X] duration:
1m0.176897355s trace:
https://jaeger.dev.storj.io/trace/6a17ba18ebe0d066
FAIL (2026-01-16 13:31:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
8945]: Uploading piece...
Jan 16 13:30:29.749952488 (21h34m26.643661098s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751054347 (21h34m26.644762956s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.751245671 (21h34m26.64495428s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752452509 (21h34m26.646161118s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.752841329 (21h34m26.646549968s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.753827407 (21h34m26.647536026s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754572565 (21h34m26.648281184s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.754930387 (21h34m26.648638996s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:29.756028858 (21h34m26.649737467s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Uploading piece...
Jan 16 13:30:30.579920945 (21h34m27.473629554s) [upload=18945]: Next returned nil segment
Jan 16 13:30:30.579958195 (21h34m27.473666804s) [upload=18945]: Waiting for error group managing segments...
Jan 16 13:30:31.002558571 (21h34m27.89626718s) [noise=true , address=81.171.14.70:4039 , piece_id=BLGME4RDK7P7TRQADPZ7GB2KTE5PZLEKL463GEU3VUICR4GVI25Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.011159918 (21h34m27.904868527s) [noise=true , address=81.171.14.70:4036 , piece_id=4EIXDXC72X3H53EOWVQOOVF3XARPLH7JXYONWN3CTYCLA5QVQLMA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.018823630 (21h34m27.912532239s) [noise=true , address=81.171.14.70:4031 , piece_id=RI6PTOKSIGRDPNZQO2A2VHMFL5A54JEADQURFNBFB5VA5OORD7GA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.035496365 (21h34m27.929204974s) [noise=true , address=81.171.14.70:4047 , piece_id=NTSJ5TNJ3M7EPTU5Y5E5UXVCPWSZWN7XBH27SFMUM6QBLQV6YIIQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.038919763 (21h34m27.932628372s) [noise=true , address=81.171.14.70:4043 , piece_id=LBOFANS3LFIOTFCOWQWWBBIULDONQFNZCIG7AOBAJ62IXQKA5Q3A, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.301468601 (21h34m28.19517722s) [noise=true , address=185.167.204.206:4038 , piece_id=KWMWYD77WHUTXSH4KT5T5RRE7IGTWCSZDGI6FS6BKAD7ZN3MWTLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312649390 (21h34m28.206358009s) [noise=true , address=185.167.204.206:4064 , piece_id=BG2U4AZGXQJHDKZ3F2JVII6JWPNSNYVVQHWYCP4Q3DPBRHRKKQ6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.312682844 (21h34m28.206391463s) [noise=true , address=185.167.204.206:4063 , piece_id=27SXYWCQLWDIIJBWE356TXK7F2TZQB6TUI523I5YDQ7VLLGROMEA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.315691953 (21h34m28.209400572s) [noise=true , address=185.167.204.205:4009 , piece_id=XFVI7TKTHXFHGVEJXYFN6MBHLHSQRKXTT2OFMZSJMR4JUFOYODVA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.321788792 (21h34m28.215497401s) [noise=true , address=185.167.204.205:4032 , piece_id=DDEICRCERAGJKOMYFURUGOVTAPTDDHFWSBQNWS2BQMFEQP52HEQQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.331501024 (21h34m28.225209634s) [noise=true , address=185.167.204.205:4027 , piece_id=RFHI6C46ZU6H6VXTBE6ASHVHONSSLH6FS7V7E7NMEG2L7FRGIGIA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.338238573 (21h34m28.231947182s) [noise=true , address=185.167.204.205:4034 , piece_id=DYQSDM67OHGU4KJCPUVIRV5W27W5SQQW54OCVJNPVJNC7POFT57Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.342157225 (21h34m28.235865844s) [noise=true , address=185.167.204.205:4028 , piece_id=EUQEYHLLEQCRF2J53C32OXEKAYMV66R63BSCVA6LSXN7ZE5IVVLQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.343599621 (21h34m28.23730824s) [noise=true , address=185.167.204.205:4008 , piece_id=QLK4BR6SFTUGWVI6OG2OQU7MJTM3ZDJRARXRLNCD26XKIN7JMEWQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.344191979 (21h34m28.237900598s) [noise=true , address=185.167.204.206:4065 , piece_id=MSHO3ZAEMBUJWAIQQRVEO5L7SEKBKH3LKLLFZFPMUCKSRBKU6G4Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.352921981 (21h34m28.2466306s) [noise=true , address=185.167.204.205:4015 , piece_id=3A2PWIOJ2IUQT5P2TODGCIAJ7UGALJI3PEKC6MEIGJFOJFYAUZPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.422565566 (21h34m28.316274175s) [noise=true , address=81.171.14.70:4033 , piece_id=5DB6U5XYTB6KQRDDNOLBSRRASY4IM6TPUEYUA34ANSNUZP45FC6Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.633526548 (21h34m28.527235167s) [noise=true , address=89.149.201.147:4010 , piece_id=CCVGE6M6TBD54BPL7SQPGEP4SE3MV2QUXBPRVTVBWR2DWJJRJG3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.669968926 (21h34m28.563677545s) [noise=true , address=89.149.201.147:4001 , piece_id=DQW3BWUXNL6E7X64R6PRTTPVPRPQVXMGQHR7HMODNSPZMHPJMT3Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.673124455 (21h34m28.566833064s) [noise=true , address=89.149.201.147:4018 , piece_id=6AIJCF33YHL37CPIEKTUMTUK572QX44ZTGQJ6JIPC46Z7L6WKVGQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.696059464 (21h34m28.589768073s) [noise=true , address=89.149.201.147:4016 , piece_id=PJUG3LPHXFGFDDJQOQA3F3YN4J2I32C6YKEC3YICCDB724Y6ON5Q, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:31.699011364 (21h34m28.592719973s) [noise=true , address=89.149.201.147:4020 , piece_id=FQVL52LD4LHPUUV7WPMOPWZBQGPFEMGEFNBKDJ5ZUJ35MZL3N2IQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:30:32.083814761 (21h34m28.97752338s) [noise=true , address=89.149.201.147:4014 , piece_id=Z2UMD4EANQOHNUP2IJLYZOUFD6335EVBHKZ4TF5CNJQYO2XH6VRQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: <nil>
Jan 16 13:31:29.083387385 (21h35m25.977095994s) [noise=true , address=89.149.201.147:4019 , piece_id=AZFUTSE5XX2YLFNQ6XD74UPYT6JD4ERQO4RWIJNVY4YRDJBRFIWA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57458->89.149.201.147:4019: use of closed network connection
Jan 16 13:31:29.083402855 (21h35m25.977111464s) [noise=true , address=89.149.201.147:4005 , piece_id=YQIVIZ5XH474SMBAOJ23KBHESN4XOPW2GJETJM5NMOTYB3ZJLKPA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34474->89.149.201.147:4005: use of closed network connection
Jan 16 13:31:29.083516611 (21h35m25.97722522s) [noise=true , address=89.149.201.147:4012 , piece_id=TS5SMG3RF6NAXFCCAS4HK33YYTUKVSTO4PNPVLX5LC5XL3I6FUSQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083538183 (21h35m25.977246792s) [noise=true , address=89.149.201.147:4015 , piece_id=752FYISBF27WKZIETWWHRGZ3L22R4XN4GNJFSNO3LCQVUQ5SYVTA, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: read tcp 188.166.225.170:48884->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:31:29.083605922 (21h35m25.977314531s) [noise=true , address=89.149.201.147:4003 , piece_id=R2GMZPZ2PEZUO4KJ24EAJS6UVP5UGIANVAMXO7KU36OXAPR7GNVQ, seg_pos={0 0}, upload=18945]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:31:29.083625889 (21h35m25.977334498s) [seg_pos={0 0}, upload=18945]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 16 13:31:29.258723876 (21h35m26.152432485s) [upload=18945]: Done uploading segments...
Jan 16 13:31:29.258758462 (21h35m26.152467071s) [upload=18945]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 1m0.176919698s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
45 out of 60 is failed (threshold=3) [X..X..XXX..XXXXXXXXXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXX] duration:
30.000599458s trace:
https://jaeger.dev.storj.io/trace/4f680aaa4ee30b72
FAIL (2026-01-16 13:31:25 UTC)
orbiter warning: context deadline exceeded
Duration: 30.000619977s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
13.812833198s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:31:12 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 13.812856732s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.002082162s trace:
https://jaeger.dev.storj.io/trace/2070d86ca05f67f7
FAIL (2026-01-16 13:30:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.002108521s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [...........XX.....X..XX..X.............X.X............X.....] duration:
14.204387868s trace:
https://jaeger.dev.storj.io/trace/78cb4176e79a71ef
FAIL (2026-01-16 13:30:33 UTC)
orbiter warning: context deadline exceeded
Duration: 14.204401425s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
15.15358445s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:30:25 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 15.1536109s
edge
gateway
us-select-1
|