Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [X.............X.X............X.............X..X..X...X......] duration:
43.039610351s trace:
https://jaeger.dev.storj.io/trace/6424ce66c661383f
FAIL (2026-01-16 13:51:59 UTC)
orbiter warning: context deadline exceeded
Duration: 43.039639737s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001280523s trace:
https://jaeger.dev.storj.io/trace/7a978b74e916c22b
FAIL (2026-01-16 13:51:49 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001345226s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001215252s trace:
https://jaeger.dev.storj.io/trace/633647c731453f9e
FAIL (2026-01-16 13:51:21 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001304703s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
7 out of 60 is failed (threshold=3) [....................X.............X........X..X.XX.....X....] duration:
5.492742484s trace:
https://jaeger.dev.storj.io/trace/18ea158581539e05
FAIL (2026-01-16 13:51:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
: Uploading piece...
Jan 16 13:47:34.817121271 (21h51m31.71082988s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817237642 (21h51m31.710946251s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817351119 (21h51m31.711059728s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817710643 (21h51m31.711419282s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817769405 (21h51m31.711478014s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818062553 (21h51m31.711771162s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818137756 (21h51m31.711846365s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818415194 (21h51m31.712123793s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818739812 (21h51m31.712448421s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819043791 (21h51m31.7127524s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819337630 (21h51m31.713046239s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819543422 (21h51m31.713252031s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819810982 (21h51m31.713519591s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:35.646384760 (21h51m32.540093369s) [upload=19196]: Next returned nil segment
Jan 16 13:47:35.646399467 (21h51m32.540108076s) [upload=19196]: Waiting for error group managing segments...
Jan 16 13:47:36.044357389 (21h51m32.938065998s) [noise=true , address=81.171.14.70:4045 , piece_id=GC5TS2METLE6BBQQGYRMWPTMVH3RMRKQJA2RXKQFCCTCJB2JXMBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.049970457 (21h51m32.943679076s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.057550509 (21h51m32.951259118s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.093832752 (21h51m32.987541361s) [noise=true , address=81.171.14.70:4026 , piece_id=YZUBTL2WRUNDUPW376BG676VAWOU2GMIMQITAKTWIKJ3HKZXNLZA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.370239906 (21h51m33.263948515s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.384125184 (21h51m33.277833793s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391293162 (21h51m33.285001771s) [noise=true , address=185.167.204.206:4064 , piece_id=2PDTBJHQW5AHYD2SOM6Q5D6BYGQJCWSWQNLRXCNRPWYBHKBS6B3Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391398192 (21h51m33.285106801s) [noise=true , address=185.167.204.206:4040 , piece_id=XCGSNFY225UO7MYFSYWQQLVMP73RIUVII7ZYTTGWZ3TQU2DPMOTA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.392101912 (21h51m33.285810521s) [noise=true , address=185.167.204.206:4061 , piece_id=EN3JISM5WLYJWU3UG3BFTNCBOP7MTIFJBRYU4KTKDFVSWCEJRVJA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.400384883 (21h51m33.294093492s) [noise=true , address=185.167.204.206:4039 , piece_id=BJWZY2S2EZSQO6CJM25WV4RIYTMS3P4CG5RHC44JXDMUGA3Z3JEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.405248333 (21h51m33.298956942s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.414198665 (21h51m33.307907274s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423293562 (21h51m33.317002171s) [noise=true , address=185.167.204.206:4045 , piece_id=FHIBGSVU2OZBBQETRZNJK6QW7DTCV7ITCEMUGEYQAN6ZYMQQYTOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423504314 (21h51m33.317212923s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.435923832 (21h51m33.329632461s) [noise=true , address=185.167.204.205:4019 , piece_id=NGOWYQ3TZHPLUJ73WVIC3YYL65EYJVBKOPUWCI4DLEGRPQIECBBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.436584970 (21h51m33.330293579s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.491250847 (21h51m33.384959456s) [noise=true , address=81.171.14.70:4031 , piece_id=EF67IQMYFBPUP6UZXDGSQLHR6X4G6OSCKEH4DYWCKI4UIHO4UVDA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.518740798 (21h51m33.412449407s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.526006844 (21h51m33.419715453s) [noise=true , address=81.171.14.70:4025 , piece_id=DCILR24FF45JKLU36SHTMAJLY3NTFNGT4UWNFJXATADZJY2W3TBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.564445552 (21h51m33.458154161s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.587122821 (21h51m33.48083146s) [noise=true , address=81.171.14.70:4038 , piece_id=VAUCLB55KGVJVL4BSWY4ZY2747BDYFXI2RK3JMPDO4ZOCVOT43EA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.679729699 (21h51m33.573438309s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.775377568 (21h51m33.669086177s) [noise=true , address=89.149.201.147:4010 , piece_id=WBP6VM7X2ZISV4K4SE6LL3ZSPYHJQK24IOGILVH6S2ZLWRRJVN7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.160795473 (21h51m34.054504082s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.251044932 (21h51m34.144753541s) [noise=true , address=89.149.201.147:4007 , piece_id=J4W7LHO6YZCI3KPZ3F3DN6P4EAV5RIMRQ2X7ZOKLAAQVNGD7HVMA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:48:34.164143586 (21h52m31.057852195s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48732->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:48:34.164198381 (21h52m31.05790699s) [noise=true , address=89.149.201.147:4020 , piece_id=CQAAHSLJFZE4GHOKB5UO7A2ANJ6N4HHFZDQ33WV25JF76L7JMMDQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic, address:89.149.201.147:4020): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:48:34.164358165 (21h52m31.058066774s) [noise=true , address=89.149.201.147:4014 , piece_id=BNGVISOX55CNKPIHIJJCXFNIMGW5FCE6GWEY4QKLIAWMVYMRAYFA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: context deadline exceeded; context deadline exceeded
Jan 16 13:48:34.164382872 (21h52m31.058091481s) [seg_pos={0 0}, upload=19196]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Jan 16 13:48:34.330174768 (21h52m31.223883387s) [upload=19196]: Done uploading segments...
Jan 16 13:48:34.330206869 (21h52m31.223915478s) [upload=19196]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.49277198s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
18.353027675s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:51:09 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 18.353044939s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [.X.............X.X............X.............X..X..X...X.....] duration:
23.444326421s trace:
https://jaeger.dev.storj.io/trace/6424ce66c661383f
FAIL (2026-01-16 13:51:04 UTC)
orbiter warning: context deadline exceeded
Duration: 23.444374353s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
17.177761401s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:50:50 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 17.177777983s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [X.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001064678s trace:
https://jaeger.dev.storj.io/trace/4e29b063e41cf1d1
FAIL (2026-01-16 13:50:49 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001088292s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
7 out of 60 is failed (threshold=3) [.....................X.............X........X..X.XX.....X...] duration:
4.647098635s trace:
https://jaeger.dev.storj.io/trace/18ea158581539e05
FAIL (2026-01-16 13:50:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
: Uploading piece...
Jan 16 13:47:34.817121271 (21h51m31.71082988s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817237642 (21h51m31.710946251s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817351119 (21h51m31.711059728s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817710643 (21h51m31.711419282s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817769405 (21h51m31.711478014s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818062553 (21h51m31.711771162s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818137756 (21h51m31.711846365s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818415194 (21h51m31.712123793s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818739812 (21h51m31.712448421s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819043791 (21h51m31.7127524s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819337630 (21h51m31.713046239s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819543422 (21h51m31.713252031s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819810982 (21h51m31.713519591s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:35.646384760 (21h51m32.540093369s) [upload=19196]: Next returned nil segment
Jan 16 13:47:35.646399467 (21h51m32.540108076s) [upload=19196]: Waiting for error group managing segments...
Jan 16 13:47:36.044357389 (21h51m32.938065998s) [noise=true , address=81.171.14.70:4045 , piece_id=GC5TS2METLE6BBQQGYRMWPTMVH3RMRKQJA2RXKQFCCTCJB2JXMBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.049970457 (21h51m32.943679076s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.057550509 (21h51m32.951259118s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.093832752 (21h51m32.987541361s) [noise=true , address=81.171.14.70:4026 , piece_id=YZUBTL2WRUNDUPW376BG676VAWOU2GMIMQITAKTWIKJ3HKZXNLZA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.370239906 (21h51m33.263948515s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.384125184 (21h51m33.277833793s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391293162 (21h51m33.285001771s) [noise=true , address=185.167.204.206:4064 , piece_id=2PDTBJHQW5AHYD2SOM6Q5D6BYGQJCWSWQNLRXCNRPWYBHKBS6B3Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391398192 (21h51m33.285106801s) [noise=true , address=185.167.204.206:4040 , piece_id=XCGSNFY225UO7MYFSYWQQLVMP73RIUVII7ZYTTGWZ3TQU2DPMOTA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.392101912 (21h51m33.285810521s) [noise=true , address=185.167.204.206:4061 , piece_id=EN3JISM5WLYJWU3UG3BFTNCBOP7MTIFJBRYU4KTKDFVSWCEJRVJA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.400384883 (21h51m33.294093492s) [noise=true , address=185.167.204.206:4039 , piece_id=BJWZY2S2EZSQO6CJM25WV4RIYTMS3P4CG5RHC44JXDMUGA3Z3JEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.405248333 (21h51m33.298956942s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.414198665 (21h51m33.307907274s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423293562 (21h51m33.317002171s) [noise=true , address=185.167.204.206:4045 , piece_id=FHIBGSVU2OZBBQETRZNJK6QW7DTCV7ITCEMUGEYQAN6ZYMQQYTOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423504314 (21h51m33.317212923s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.435923832 (21h51m33.329632461s) [noise=true , address=185.167.204.205:4019 , piece_id=NGOWYQ3TZHPLUJ73WVIC3YYL65EYJVBKOPUWCI4DLEGRPQIECBBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.436584970 (21h51m33.330293579s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.491250847 (21h51m33.384959456s) [noise=true , address=81.171.14.70:4031 , piece_id=EF67IQMYFBPUP6UZXDGSQLHR6X4G6OSCKEH4DYWCKI4UIHO4UVDA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.518740798 (21h51m33.412449407s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.526006844 (21h51m33.419715453s) [noise=true , address=81.171.14.70:4025 , piece_id=DCILR24FF45JKLU36SHTMAJLY3NTFNGT4UWNFJXATADZJY2W3TBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.564445552 (21h51m33.458154161s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.587122821 (21h51m33.48083146s) [noise=true , address=81.171.14.70:4038 , piece_id=VAUCLB55KGVJVL4BSWY4ZY2747BDYFXI2RK3JMPDO4ZOCVOT43EA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.679729699 (21h51m33.573438309s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.775377568 (21h51m33.669086177s) [noise=true , address=89.149.201.147:4010 , piece_id=WBP6VM7X2ZISV4K4SE6LL3ZSPYHJQK24IOGILVH6S2ZLWRRJVN7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.160795473 (21h51m34.054504082s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.251044932 (21h51m34.144753541s) [noise=true , address=89.149.201.147:4007 , piece_id=J4W7LHO6YZCI3KPZ3F3DN6P4EAV5RIMRQ2X7ZOKLAAQVNGD7HVMA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:48:34.164143586 (21h52m31.057852195s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48732->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:48:34.164198381 (21h52m31.05790699s) [noise=true , address=89.149.201.147:4020 , piece_id=CQAAHSLJFZE4GHOKB5UO7A2ANJ6N4HHFZDQ33WV25JF76L7JMMDQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic, address:89.149.201.147:4020): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:48:34.164358165 (21h52m31.058066774s) [noise=true , address=89.149.201.147:4014 , piece_id=BNGVISOX55CNKPIHIJJCXFNIMGW5FCE6GWEY4QKLIAWMVYMRAYFA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: context deadline exceeded; context deadline exceeded
Jan 16 13:48:34.164382872 (21h52m31.058091481s) [seg_pos={0 0}, upload=19196]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Jan 16 13:48:34.330174768 (21h52m31.223883387s) [upload=19196]: Done uploading segments...
Jan 16 13:48:34.330206869 (21h52m31.223915478s) [upload=19196]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 4.647124396s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001153123s trace:
https://jaeger.dev.storj.io/trace/65efdf0c08b3ea27
FAIL (2026-01-16 13:50:20 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001174675s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001751213s trace:
https://jaeger.dev.storj.io/trace/50fd71ce948590e2
FAIL (2026-01-16 13:49:58 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001771061s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
12.50254683s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:49:56 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 12.502627385s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [..X.............X.X............X.............X..X..X...X....] duration:
14.941749531s trace:
https://jaeger.dev.storj.io/trace/6424ce66c661383f
FAIL (2026-01-16 13:49:48 UTC)
orbiter warning: context deadline exceeded
Duration: 14.941773929s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000707164s trace:
https://jaeger.dev.storj.io/trace/67835faa755366a0
FAIL (2026-01-16 13:49:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000728004s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
7 out of 60 is failed (threshold=3) [......................X.............X........X..X.XX.....X..] duration:
5.336642172s trace:
https://jaeger.dev.storj.io/trace/18ea158581539e05
FAIL (2026-01-16 13:49:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
: Uploading piece...
Jan 16 13:47:34.817121271 (21h51m31.71082988s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817237642 (21h51m31.710946251s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817351119 (21h51m31.711059728s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817710643 (21h51m31.711419282s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817769405 (21h51m31.711478014s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818062553 (21h51m31.711771162s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818137756 (21h51m31.711846365s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818415194 (21h51m31.712123793s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818739812 (21h51m31.712448421s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819043791 (21h51m31.7127524s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819337630 (21h51m31.713046239s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819543422 (21h51m31.713252031s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819810982 (21h51m31.713519591s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:35.646384760 (21h51m32.540093369s) [upload=19196]: Next returned nil segment
Jan 16 13:47:35.646399467 (21h51m32.540108076s) [upload=19196]: Waiting for error group managing segments...
Jan 16 13:47:36.044357389 (21h51m32.938065998s) [noise=true , address=81.171.14.70:4045 , piece_id=GC5TS2METLE6BBQQGYRMWPTMVH3RMRKQJA2RXKQFCCTCJB2JXMBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.049970457 (21h51m32.943679076s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.057550509 (21h51m32.951259118s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.093832752 (21h51m32.987541361s) [noise=true , address=81.171.14.70:4026 , piece_id=YZUBTL2WRUNDUPW376BG676VAWOU2GMIMQITAKTWIKJ3HKZXNLZA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.370239906 (21h51m33.263948515s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.384125184 (21h51m33.277833793s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391293162 (21h51m33.285001771s) [noise=true , address=185.167.204.206:4064 , piece_id=2PDTBJHQW5AHYD2SOM6Q5D6BYGQJCWSWQNLRXCNRPWYBHKBS6B3Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391398192 (21h51m33.285106801s) [noise=true , address=185.167.204.206:4040 , piece_id=XCGSNFY225UO7MYFSYWQQLVMP73RIUVII7ZYTTGWZ3TQU2DPMOTA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.392101912 (21h51m33.285810521s) [noise=true , address=185.167.204.206:4061 , piece_id=EN3JISM5WLYJWU3UG3BFTNCBOP7MTIFJBRYU4KTKDFVSWCEJRVJA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.400384883 (21h51m33.294093492s) [noise=true , address=185.167.204.206:4039 , piece_id=BJWZY2S2EZSQO6CJM25WV4RIYTMS3P4CG5RHC44JXDMUGA3Z3JEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.405248333 (21h51m33.298956942s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.414198665 (21h51m33.307907274s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423293562 (21h51m33.317002171s) [noise=true , address=185.167.204.206:4045 , piece_id=FHIBGSVU2OZBBQETRZNJK6QW7DTCV7ITCEMUGEYQAN6ZYMQQYTOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423504314 (21h51m33.317212923s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.435923832 (21h51m33.329632461s) [noise=true , address=185.167.204.205:4019 , piece_id=NGOWYQ3TZHPLUJ73WVIC3YYL65EYJVBKOPUWCI4DLEGRPQIECBBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.436584970 (21h51m33.330293579s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.491250847 (21h51m33.384959456s) [noise=true , address=81.171.14.70:4031 , piece_id=EF67IQMYFBPUP6UZXDGSQLHR6X4G6OSCKEH4DYWCKI4UIHO4UVDA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.518740798 (21h51m33.412449407s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.526006844 (21h51m33.419715453s) [noise=true , address=81.171.14.70:4025 , piece_id=DCILR24FF45JKLU36SHTMAJLY3NTFNGT4UWNFJXATADZJY2W3TBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.564445552 (21h51m33.458154161s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.587122821 (21h51m33.48083146s) [noise=true , address=81.171.14.70:4038 , piece_id=VAUCLB55KGVJVL4BSWY4ZY2747BDYFXI2RK3JMPDO4ZOCVOT43EA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.679729699 (21h51m33.573438309s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.775377568 (21h51m33.669086177s) [noise=true , address=89.149.201.147:4010 , piece_id=WBP6VM7X2ZISV4K4SE6LL3ZSPYHJQK24IOGILVH6S2ZLWRRJVN7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.160795473 (21h51m34.054504082s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.251044932 (21h51m34.144753541s) [noise=true , address=89.149.201.147:4007 , piece_id=J4W7LHO6YZCI3KPZ3F3DN6P4EAV5RIMRQ2X7ZOKLAAQVNGD7HVMA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:48:34.164143586 (21h52m31.057852195s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48732->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:48:34.164198381 (21h52m31.05790699s) [noise=true , address=89.149.201.147:4020 , piece_id=CQAAHSLJFZE4GHOKB5UO7A2ANJ6N4HHFZDQ33WV25JF76L7JMMDQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic, address:89.149.201.147:4020): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:48:34.164358165 (21h52m31.058066774s) [noise=true , address=89.149.201.147:4014 , piece_id=BNGVISOX55CNKPIHIJJCXFNIMGW5FCE6GWEY4QKLIAWMVYMRAYFA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: context deadline exceeded; context deadline exceeded
Jan 16 13:48:34.164382872 (21h52m31.058091481s) [seg_pos={0 0}, upload=19196]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Jan 16 13:48:34.330174768 (21h52m31.223883387s) [upload=19196]: Done uploading segments...
Jan 16 13:48:34.330206869 (21h52m31.223915478s) [upload=19196]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.336672839s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
13.165578636s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:49:24 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 13.165601648s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [X..X.............X.X............X.............X..X..X...X...] duration:
45.671023648s trace:
https://jaeger.dev.storj.io/trace/6424ce66c661383f
FAIL (2026-01-16 13:49:18 UTC)
orbiter warning: context deadline exceeded
Duration: 45.6710508s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001192522s trace:
https://jaeger.dev.storj.io/trace/5d4e5f766ca2287
FAIL (2026-01-16 13:49:17 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001211327s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
7 out of 60 is failed (threshold=3) [.......................X.............X........X..X.XX.....X.] duration:
4.868800469s trace:
https://jaeger.dev.storj.io/trace/18ea158581539e05
FAIL (2026-01-16 13:48:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
: Uploading piece...
Jan 16 13:47:34.817121271 (21h51m31.71082988s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817237642 (21h51m31.710946251s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817351119 (21h51m31.711059728s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817710643 (21h51m31.711419282s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817769405 (21h51m31.711478014s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818062553 (21h51m31.711771162s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818137756 (21h51m31.711846365s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818415194 (21h51m31.712123793s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818739812 (21h51m31.712448421s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819043791 (21h51m31.7127524s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819337630 (21h51m31.713046239s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819543422 (21h51m31.713252031s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819810982 (21h51m31.713519591s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:35.646384760 (21h51m32.540093369s) [upload=19196]: Next returned nil segment
Jan 16 13:47:35.646399467 (21h51m32.540108076s) [upload=19196]: Waiting for error group managing segments...
Jan 16 13:47:36.044357389 (21h51m32.938065998s) [noise=true , address=81.171.14.70:4045 , piece_id=GC5TS2METLE6BBQQGYRMWPTMVH3RMRKQJA2RXKQFCCTCJB2JXMBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.049970457 (21h51m32.943679076s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.057550509 (21h51m32.951259118s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.093832752 (21h51m32.987541361s) [noise=true , address=81.171.14.70:4026 , piece_id=YZUBTL2WRUNDUPW376BG676VAWOU2GMIMQITAKTWIKJ3HKZXNLZA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.370239906 (21h51m33.263948515s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.384125184 (21h51m33.277833793s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391293162 (21h51m33.285001771s) [noise=true , address=185.167.204.206:4064 , piece_id=2PDTBJHQW5AHYD2SOM6Q5D6BYGQJCWSWQNLRXCNRPWYBHKBS6B3Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391398192 (21h51m33.285106801s) [noise=true , address=185.167.204.206:4040 , piece_id=XCGSNFY225UO7MYFSYWQQLVMP73RIUVII7ZYTTGWZ3TQU2DPMOTA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.392101912 (21h51m33.285810521s) [noise=true , address=185.167.204.206:4061 , piece_id=EN3JISM5WLYJWU3UG3BFTNCBOP7MTIFJBRYU4KTKDFVSWCEJRVJA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.400384883 (21h51m33.294093492s) [noise=true , address=185.167.204.206:4039 , piece_id=BJWZY2S2EZSQO6CJM25WV4RIYTMS3P4CG5RHC44JXDMUGA3Z3JEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.405248333 (21h51m33.298956942s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.414198665 (21h51m33.307907274s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423293562 (21h51m33.317002171s) [noise=true , address=185.167.204.206:4045 , piece_id=FHIBGSVU2OZBBQETRZNJK6QW7DTCV7ITCEMUGEYQAN6ZYMQQYTOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423504314 (21h51m33.317212923s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.435923832 (21h51m33.329632461s) [noise=true , address=185.167.204.205:4019 , piece_id=NGOWYQ3TZHPLUJ73WVIC3YYL65EYJVBKOPUWCI4DLEGRPQIECBBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.436584970 (21h51m33.330293579s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.491250847 (21h51m33.384959456s) [noise=true , address=81.171.14.70:4031 , piece_id=EF67IQMYFBPUP6UZXDGSQLHR6X4G6OSCKEH4DYWCKI4UIHO4UVDA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.518740798 (21h51m33.412449407s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.526006844 (21h51m33.419715453s) [noise=true , address=81.171.14.70:4025 , piece_id=DCILR24FF45JKLU36SHTMAJLY3NTFNGT4UWNFJXATADZJY2W3TBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.564445552 (21h51m33.458154161s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.587122821 (21h51m33.48083146s) [noise=true , address=81.171.14.70:4038 , piece_id=VAUCLB55KGVJVL4BSWY4ZY2747BDYFXI2RK3JMPDO4ZOCVOT43EA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.679729699 (21h51m33.573438309s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.775377568 (21h51m33.669086177s) [noise=true , address=89.149.201.147:4010 , piece_id=WBP6VM7X2ZISV4K4SE6LL3ZSPYHJQK24IOGILVH6S2ZLWRRJVN7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.160795473 (21h51m34.054504082s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.251044932 (21h51m34.144753541s) [noise=true , address=89.149.201.147:4007 , piece_id=J4W7LHO6YZCI3KPZ3F3DN6P4EAV5RIMRQ2X7ZOKLAAQVNGD7HVMA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:48:34.164143586 (21h52m31.057852195s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48732->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:48:34.164198381 (21h52m31.05790699s) [noise=true , address=89.149.201.147:4020 , piece_id=CQAAHSLJFZE4GHOKB5UO7A2ANJ6N4HHFZDQ33WV25JF76L7JMMDQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic, address:89.149.201.147:4020): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:48:34.164358165 (21h52m31.058066774s) [noise=true , address=89.149.201.147:4014 , piece_id=BNGVISOX55CNKPIHIJJCXFNIMGW5FCE6GWEY4QKLIAWMVYMRAYFA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: context deadline exceeded; context deadline exceeded
Jan 16 13:48:34.164382872 (21h52m31.058091481s) [seg_pos={0 0}, upload=19196]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Jan 16 13:48:34.330174768 (21h52m31.223883387s) [upload=19196]: Done uploading segments...
Jan 16 13:48:34.330206869 (21h52m31.223915478s) [upload=19196]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 4.868831559s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
7 out of 60 is failed (threshold=3) [........................X.............X........X..X.XX.....X] duration:
1m0.168565665s trace:
https://jaeger.dev.storj.io/trace/18ea158581539e05
FAIL (2026-01-16 13:48:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
: Uploading piece...
Jan 16 13:47:34.817121271 (21h51m31.71082988s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817237642 (21h51m31.710946251s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817351119 (21h51m31.711059728s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817710643 (21h51m31.711419282s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.817769405 (21h51m31.711478014s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818062553 (21h51m31.711771162s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818137756 (21h51m31.711846365s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818415194 (21h51m31.712123793s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.818739812 (21h51m31.712448421s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819043791 (21h51m31.7127524s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819337630 (21h51m31.713046239s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819543422 (21h51m31.713252031s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:34.819810982 (21h51m31.713519591s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Uploading piece...
Jan 16 13:47:35.646384760 (21h51m32.540093369s) [upload=19196]: Next returned nil segment
Jan 16 13:47:35.646399467 (21h51m32.540108076s) [upload=19196]: Waiting for error group managing segments...
Jan 16 13:47:36.044357389 (21h51m32.938065998s) [noise=true , address=81.171.14.70:4045 , piece_id=GC5TS2METLE6BBQQGYRMWPTMVH3RMRKQJA2RXKQFCCTCJB2JXMBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.049970457 (21h51m32.943679076s) [noise=true , address=81.171.14.70:4042 , piece_id=PFKTRW3DI6O7EQ4LSDBQM5YQDJ6JQLXCNZTBCNMTYD3SKQWVGGLQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.057550509 (21h51m32.951259118s) [noise=true , address=81.171.14.70:4040 , piece_id=4ZRIS4UHZC6SMLPEIXS6KU6UAKYVIGI6ZMJFRFJ25PY4RB32WVEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.093832752 (21h51m32.987541361s) [noise=true , address=81.171.14.70:4026 , piece_id=YZUBTL2WRUNDUPW376BG676VAWOU2GMIMQITAKTWIKJ3HKZXNLZA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.370239906 (21h51m33.263948515s) [noise=true , address=185.167.204.206:4067 , piece_id=BSRJBOCNWZIYTEP6HY3NOGZORQTPCWKN43E6HPVC37LY5I5CJYNA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.384125184 (21h51m33.277833793s) [noise=true , address=185.167.204.206:4062 , piece_id=3Z6VWEXXIA3ISY73C5JV5S5ROLCBR6DSQWZVI6Q3QSSYLRBNM6LQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391293162 (21h51m33.285001771s) [noise=true , address=185.167.204.206:4064 , piece_id=2PDTBJHQW5AHYD2SOM6Q5D6BYGQJCWSWQNLRXCNRPWYBHKBS6B3Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.391398192 (21h51m33.285106801s) [noise=true , address=185.167.204.206:4040 , piece_id=XCGSNFY225UO7MYFSYWQQLVMP73RIUVII7ZYTTGWZ3TQU2DPMOTA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.392101912 (21h51m33.285810521s) [noise=true , address=185.167.204.206:4061 , piece_id=EN3JISM5WLYJWU3UG3BFTNCBOP7MTIFJBRYU4KTKDFVSWCEJRVJA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.400384883 (21h51m33.294093492s) [noise=true , address=185.167.204.206:4039 , piece_id=BJWZY2S2EZSQO6CJM25WV4RIYTMS3P4CG5RHC44JXDMUGA3Z3JEQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.405248333 (21h51m33.298956942s) [noise=true , address=185.167.204.206:4050 , piece_id=E5FLLMTDTPGGIAKZDIIWPBOFWQL7YXZSUA7Z5YSN2KWJOASDJF6Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.414198665 (21h51m33.307907274s) [noise=true , address=185.167.204.205:4006 , piece_id=XCZ2NHQZKDA7AADK3JP26Y3WOZMXQ5IJV5THD5V2P5TX3F7WAFOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423293562 (21h51m33.317002171s) [noise=true , address=185.167.204.206:4045 , piece_id=FHIBGSVU2OZBBQETRZNJK6QW7DTCV7ITCEMUGEYQAN6ZYMQQYTOQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.423504314 (21h51m33.317212923s) [noise=true , address=185.167.204.205:4018 , piece_id=3J3LKBXHO5E2IYUVRO7NG7BSYXYBQEFRHV23OQHXHFBYBIYOBQ2A, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.435923832 (21h51m33.329632461s) [noise=true , address=185.167.204.205:4019 , piece_id=NGOWYQ3TZHPLUJ73WVIC3YYL65EYJVBKOPUWCI4DLEGRPQIECBBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.436584970 (21h51m33.330293579s) [noise=true , address=185.167.204.205:4026 , piece_id=SVVM4Q6OCLSV7TWVURRBYN765J44OTG7AFOQS3RZX7GKFMDJ6ZJQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.491250847 (21h51m33.384959456s) [noise=true , address=81.171.14.70:4031 , piece_id=EF67IQMYFBPUP6UZXDGSQLHR6X4G6OSCKEH4DYWCKI4UIHO4UVDA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.518740798 (21h51m33.412449407s) [noise=true , address=81.171.14.70:4030 , piece_id=SLIXLC6YJ5PLS4MCXJIACYV5MIFOQSBARATYC3WAEZEZSVWICMCQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.526006844 (21h51m33.419715453s) [noise=true , address=81.171.14.70:4025 , piece_id=DCILR24FF45JKLU36SHTMAJLY3NTFNGT4UWNFJXATADZJY2W3TBA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.564445552 (21h51m33.458154161s) [noise=true , address=81.171.14.70:4027 , piece_id=7YO3ZAQFFZOSWSG2JH3KOHWDEFQJLOCCQ5FUGSIT7UUREHROOGYA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.587122821 (21h51m33.48083146s) [noise=true , address=81.171.14.70:4038 , piece_id=VAUCLB55KGVJVL4BSWY4ZY2747BDYFXI2RK3JMPDO4ZOCVOT43EA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.679729699 (21h51m33.573438309s) [noise=true , address=89.149.201.147:4012 , piece_id=SPAZ2DY75JXHC2PWPRI3QG7HYCGHM4GVXN5DY5ZWYDTM6637EV7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:36.775377568 (21h51m33.669086177s) [noise=true , address=89.149.201.147:4010 , piece_id=WBP6VM7X2ZISV4K4SE6LL3ZSPYHJQK24IOGILVH6S2ZLWRRJVN7Q, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.160795473 (21h51m34.054504082s) [noise=true , address=89.149.201.147:4023 , piece_id=LQXOUV6UVIHCQTXPVMC7NYINBLEZWF4H7W6BSOWGK6BFG5ARQ3VQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:47:37.251044932 (21h51m34.144753541s) [noise=true , address=89.149.201.147:4007 , piece_id=J4W7LHO6YZCI3KPZ3F3DN6P4EAV5RIMRQ2X7ZOKLAAQVNGD7HVMA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: <nil>
Jan 16 13:48:34.164143586 (21h52m31.057852195s) [noise=true , address=89.149.201.147:4011 , piece_id=TFRZQWAVN2NUSEWM4L33PKZKUYWQYCV3KCRHTKVTABLDZ7GSVVSQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48732->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:48:34.164198381 (21h52m31.05790699s) [noise=true , address=89.149.201.147:4020 , piece_id=CQAAHSLJFZE4GHOKB5UO7A2ANJ6N4HHFZDQ33WV25JF76L7JMMDQ, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic, address:89.149.201.147:4020): protocol: expected piece hash; context deadline exceeded; EOF
Jan 16 13:48:34.164358165 (21h52m31.058066774s) [noise=true , address=89.149.201.147:4014 , piece_id=BNGVISOX55CNKPIHIJJCXFNIMGW5FCE6GWEY4QKLIAWMVYMRAYFA, seg_pos={0 0}, upload=19196]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: context deadline exceeded; context deadline exceeded
Jan 16 13:48:34.164382872 (21h52m31.058091481s) [seg_pos={0 0}, upload=19196]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Jan 16 13:48:34.330174768 (21h52m31.223883387s) [upload=19196]: Done uploading segments...
Jan 16 13:48:34.330206869 (21h52m31.223915478s) [upload=19196]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 1m0.168591132s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001254689s trace:
https://jaeger.dev.storj.io/trace/5405de1b83cece53
FAIL (2026-01-16 13:48:33 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001278884s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [XX..X.............X.X............X.............X..X..X...X..] duration:
15.069800719s trace:
https://jaeger.dev.storj.io/trace/6424ce66c661383f
FAIL (2026-01-16 13:48:32 UTC)
orbiter warning: context deadline exceeded
Duration: 15.069821678s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001709803s trace:
https://jaeger.dev.storj.io/trace/50ec73d36a6fdf9e
FAIL (2026-01-16 13:48:24 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001719793s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
14.165450361s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:48:21 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 14.165475559s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [.XX..X.............X.X............X.............X..X..X...X.] duration:
26.823158045s trace:
https://jaeger.dev.storj.io/trace/6424ce66c661383f
FAIL (2026-01-16 13:47:58 UTC)
orbiter warning: context deadline exceeded
Duration: 26.823215173s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001192519s trace:
https://jaeger.dev.storj.io/trace/598a35eba3382f51
FAIL (2026-01-16 13:47:54 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001202779s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000975876s trace:
https://jaeger.dev.storj.io/trace/550d3486be31e638
FAIL (2026-01-16 13:47:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000995914s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [..XX..X.............X.X............X.............X..X..X...X] duration:
1m0.001264334s trace:
https://jaeger.dev.storj.io/trace/6424ce66c661383f
FAIL (2026-01-16 13:47:31 UTC)
orbiter warning: context deadline exceeded
Duration: 1m0.001276326s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001309057s trace:
https://jaeger.dev.storj.io/trace/211ed86fc9476647
FAIL (2026-01-16 13:47:24 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001322844s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000861747s trace:
https://jaeger.dev.storj.io/trace/3b2bf12252312e77
FAIL (2026-01-16 13:47:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00089502s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
20.34186869s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:47:19 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 20.341896533s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.........................X.............X........X..X.XX.....] duration:
4.617644861s trace:
https://jaeger.dev.storj.io/trace/287073b7bf0613bb
FAIL (2026-01-16 13:47:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
s=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512857569 (21h47m7.406566178s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512903297 (21h47m7.406611906s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513163702 (21h47m7.406872311s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513210451 (21h47m7.40691906s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513428296 (21h47m7.407136906s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513547533 (21h47m7.407256152s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513938307 (21h47m7.407646916s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514317599 (21h47m7.408026208s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514667566 (21h47m7.408376175s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:11.336233602 (21h47m8.229942212s) [upload=19126]: Next returned nil segment
Jan 16 13:43:11.336267347 (21h47m8.229975956s) [upload=19126]: Waiting for error group managing segments...
Jan 16 13:43:11.728156896 (21h47m8.621865516s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.744681339 (21h47m8.638389948s) [noise=true , address=81.171.14.70:4031 , piece_id=I2QFNTRNW5BSEW443K4HEMRPFWDWMS4A7OGLCBGL6MOBLBIVHXGA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.781770550 (21h47m8.675479159s) [noise=true , address=81.171.14.70:4039 , piece_id=PJJUDVG42DYTROSHW2EQ37F5KXADJDEJFE6WNHHZ756KVWQHOMMQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.060761834 (21h47m8.954470453s) [noise=true , address=185.167.204.205:4033 , piece_id=WYD6DRJKUXEGOYG2JKXTSPE6MGZ3PHAGTUKFAXXD5UZMOVDJPETQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066620951 (21h47m8.96032957s) [noise=true , address=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066990785 (21h47m8.960699404s) [noise=true , address=185.167.204.205:4031 , piece_id=7QWMBCQS2HLDT6LYENQCTPGQFCYSFHDBHZCEDCLTJ3XGYBEIEIUQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.067370788 (21h47m8.961079407s) [noise=true , address=185.167.204.205:4027 , piece_id=65HE6VZ67VQWRVZ52QTIOJR4YTZVEZHW4YXLTB3I3MMACX3QU5KQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.070378284 (21h47m8.964086903s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080751365 (21h47m8.974459984s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080916139 (21h47m8.974624748s) [noise=true , address=185.167.204.205:4023 , piece_id=SP3YCSO77HYXX3XHWPWZM4KZNESTVCVQN74T4X4AZHIMGSXHYCYQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.081172958 (21h47m8.974881567s) [noise=true , address=185.167.204.205:4019 , piece_id=MOY2FK2KGUGPPJ2IT5564P4O4Y72ELKITH43E76MQNK5WBF2ZI2A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.090478015 (21h47m8.984186624s) [noise=true , address=185.167.204.205:4013 , piece_id=CAJYKOAHLSKEAAM6BR3S4G24JB7CZYAQ2MSEJBRC4PYTPSTFZD5A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.095728263 (21h47m8.989436872s) [noise=true , address=185.167.204.206:4064 , piece_id=4MG2QWPMQD6BYDRJVCBPLMAYOECLVNG3YZJ6JPK6RDKXMWMXUEFA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.097262153 (21h47m8.990970763s) [noise=true , address=185.167.204.205:4034 , piece_id=GX5ZKXOND7HFY6QSA5H4BAOS343XACNZYU7CWY5LBY3TISMJBRBQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.099826616 (21h47m8.993535225s) [noise=true , address=185.167.204.206:4058 , piece_id=QLKSPOBUDDAPJK5R2EMJP74XM6TEBQLRSCBW2D4SUDYLAF5GAWZA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.111103288 (21h47m9.004811897s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.122116057 (21h47m9.015824666s) [noise=true , address=185.167.204.205:4002 , piece_id=IF42VMTKTFOZPV7MVZ6XEZA2VZQ4W7OB6GOU56AGM4V6UCICLKKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.126871291 (21h47m9.0205799s) [noise=true , address=185.167.204.205:4015 , piece_id=FILYFJPIFRGG7GWSPSWQY2IBXAJPFNOUWGCP234KANWQJD5XPZUA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.127756125 (21h47m9.021464734s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.131797441 (21h47m9.02550605s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.152285220 (21h47m9.045993839s) [noise=true , address=185.167.204.206:4054 , piece_id=OR36BJX2U7KWL3YGZDAMRNLC44IDHRD4FXVZDP4XDLVWA73YHCIA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.154173035 (21h47m9.047881644s) [noise=true , address=185.167.204.206:4057 , piece_id=WXMMXS66XDZLHNXKIR44R5Q54CLCQN77YMKOZ2RM7MPVTXT3M67A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.397697308 (21h47m9.291405927s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.847835054 (21h47m9.741543664s) [noise=true , address=89.149.201.147:4003 , piece_id=E72MB7PBJQUMJJLOEQC4FDZO6PEZW5S2I6UH46JDPQBS7CKGHTJQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:44:09.858617030 (21h48m6.752325639s) [noise=true , address=89.149.201.147:4001 , piece_id=RBW4Y52Y7K4UPKZ6J67HOW5VN75YUFIHFRUYZYE3Y2O6WPOTIYXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection
Jan 16 13:44:09.858698235 (21h48m6.752406864s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE, address:89.149.201.147:4022): protocol: read tcp 188.166.225.170:49424->89.149.201.147:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858727541 (21h48m6.75243616s) [noise=true , address=89.149.201.147:4017 , piece_id=VOZD426Q4XUTLC7WBYGSQ54BS6VK4KP5TQMZ7R4SNYDRZXBEIW6A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection
Jan 16 13:44:09.858851426 (21h48m6.752560035s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 188.166.225.170:59878->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858875492 (21h48m6.752584101s) [seg_pos={0 0}, upload=19126]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Jan 16 13:44:10.026477265 (21h48m6.920185874s) [upload=19126]: Done uploading segments...
Jan 16 13:44:10.026510469 (21h48m6.920219088s) [upload=19126]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.617670889s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.000724696s trace:
https://jaeger.dev.storj.io/trace/3248862cb0728f1f
FAIL (2026-01-16 13:46:54 UTC)
orbiter warning: context deadline exceeded
Duration: 30.000744976s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000791002s trace:
https://jaeger.dev.storj.io/trace/7b74923780b9f98f
FAIL (2026-01-16 13:46:45 UTC)
orbiter warning: context deadline exceeded
Duration: 5.000813045s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
11.231556774s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:46:41 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 11.231583035s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [X..XX..X.............X.X............X.............X..X..X...] duration:
57.965561711s trace:
https://jaeger.dev.storj.io/trace/5475a631601811f5
FAIL (2026-01-16 13:46:31 UTC)
orbiter warning: context deadline exceeded
Duration: 57.965591276s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..........................X.............X........X..X.XX....] duration:
5.077939687s trace:
https://jaeger.dev.storj.io/trace/287073b7bf0613bb
FAIL (2026-01-16 13:46:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
s=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512857569 (21h47m7.406566178s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512903297 (21h47m7.406611906s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513163702 (21h47m7.406872311s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513210451 (21h47m7.40691906s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513428296 (21h47m7.407136906s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513547533 (21h47m7.407256152s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513938307 (21h47m7.407646916s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514317599 (21h47m7.408026208s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514667566 (21h47m7.408376175s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:11.336233602 (21h47m8.229942212s) [upload=19126]: Next returned nil segment
Jan 16 13:43:11.336267347 (21h47m8.229975956s) [upload=19126]: Waiting for error group managing segments...
Jan 16 13:43:11.728156896 (21h47m8.621865516s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.744681339 (21h47m8.638389948s) [noise=true , address=81.171.14.70:4031 , piece_id=I2QFNTRNW5BSEW443K4HEMRPFWDWMS4A7OGLCBGL6MOBLBIVHXGA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.781770550 (21h47m8.675479159s) [noise=true , address=81.171.14.70:4039 , piece_id=PJJUDVG42DYTROSHW2EQ37F5KXADJDEJFE6WNHHZ756KVWQHOMMQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.060761834 (21h47m8.954470453s) [noise=true , address=185.167.204.205:4033 , piece_id=WYD6DRJKUXEGOYG2JKXTSPE6MGZ3PHAGTUKFAXXD5UZMOVDJPETQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066620951 (21h47m8.96032957s) [noise=true , address=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066990785 (21h47m8.960699404s) [noise=true , address=185.167.204.205:4031 , piece_id=7QWMBCQS2HLDT6LYENQCTPGQFCYSFHDBHZCEDCLTJ3XGYBEIEIUQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.067370788 (21h47m8.961079407s) [noise=true , address=185.167.204.205:4027 , piece_id=65HE6VZ67VQWRVZ52QTIOJR4YTZVEZHW4YXLTB3I3MMACX3QU5KQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.070378284 (21h47m8.964086903s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080751365 (21h47m8.974459984s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080916139 (21h47m8.974624748s) [noise=true , address=185.167.204.205:4023 , piece_id=SP3YCSO77HYXX3XHWPWZM4KZNESTVCVQN74T4X4AZHIMGSXHYCYQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.081172958 (21h47m8.974881567s) [noise=true , address=185.167.204.205:4019 , piece_id=MOY2FK2KGUGPPJ2IT5564P4O4Y72ELKITH43E76MQNK5WBF2ZI2A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.090478015 (21h47m8.984186624s) [noise=true , address=185.167.204.205:4013 , piece_id=CAJYKOAHLSKEAAM6BR3S4G24JB7CZYAQ2MSEJBRC4PYTPSTFZD5A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.095728263 (21h47m8.989436872s) [noise=true , address=185.167.204.206:4064 , piece_id=4MG2QWPMQD6BYDRJVCBPLMAYOECLVNG3YZJ6JPK6RDKXMWMXUEFA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.097262153 (21h47m8.990970763s) [noise=true , address=185.167.204.205:4034 , piece_id=GX5ZKXOND7HFY6QSA5H4BAOS343XACNZYU7CWY5LBY3TISMJBRBQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.099826616 (21h47m8.993535225s) [noise=true , address=185.167.204.206:4058 , piece_id=QLKSPOBUDDAPJK5R2EMJP74XM6TEBQLRSCBW2D4SUDYLAF5GAWZA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.111103288 (21h47m9.004811897s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.122116057 (21h47m9.015824666s) [noise=true , address=185.167.204.205:4002 , piece_id=IF42VMTKTFOZPV7MVZ6XEZA2VZQ4W7OB6GOU56AGM4V6UCICLKKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.126871291 (21h47m9.0205799s) [noise=true , address=185.167.204.205:4015 , piece_id=FILYFJPIFRGG7GWSPSWQY2IBXAJPFNOUWGCP234KANWQJD5XPZUA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.127756125 (21h47m9.021464734s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.131797441 (21h47m9.02550605s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.152285220 (21h47m9.045993839s) [noise=true , address=185.167.204.206:4054 , piece_id=OR36BJX2U7KWL3YGZDAMRNLC44IDHRD4FXVZDP4XDLVWA73YHCIA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.154173035 (21h47m9.047881644s) [noise=true , address=185.167.204.206:4057 , piece_id=WXMMXS66XDZLHNXKIR44R5Q54CLCQN77YMKOZ2RM7MPVTXT3M67A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.397697308 (21h47m9.291405927s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.847835054 (21h47m9.741543664s) [noise=true , address=89.149.201.147:4003 , piece_id=E72MB7PBJQUMJJLOEQC4FDZO6PEZW5S2I6UH46JDPQBS7CKGHTJQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:44:09.858617030 (21h48m6.752325639s) [noise=true , address=89.149.201.147:4001 , piece_id=RBW4Y52Y7K4UPKZ6J67HOW5VN75YUFIHFRUYZYE3Y2O6WPOTIYXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection
Jan 16 13:44:09.858698235 (21h48m6.752406864s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE, address:89.149.201.147:4022): protocol: read tcp 188.166.225.170:49424->89.149.201.147:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858727541 (21h48m6.75243616s) [noise=true , address=89.149.201.147:4017 , piece_id=VOZD426Q4XUTLC7WBYGSQ54BS6VK4KP5TQMZ7R4SNYDRZXBEIW6A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection
Jan 16 13:44:09.858851426 (21h48m6.752560035s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 188.166.225.170:59878->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858875492 (21h48m6.752584101s) [seg_pos={0 0}, upload=19126]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Jan 16 13:44:10.026477265 (21h48m6.920185874s) [upload=19126]: Done uploading segments...
Jan 16 13:44:10.026510469 (21h48m6.920219088s) [upload=19126]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 5.0779627s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
15.014087041s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:46:02 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 15.014132107s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001158964s trace:
https://jaeger.dev.storj.io/trace/ef75708e38a1e20
FAIL (2026-01-16 13:46:00 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001185043s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.001267138s trace:
https://jaeger.dev.storj.io/trace/236ada71586f67bd
FAIL (2026-01-16 13:45:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001288159s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...........................X.............X........X..X.XX...] duration:
4.670503657s trace:
https://jaeger.dev.storj.io/trace/287073b7bf0613bb
FAIL (2026-01-16 13:45:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
s=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512857569 (21h47m7.406566178s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512903297 (21h47m7.406611906s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513163702 (21h47m7.406872311s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513210451 (21h47m7.40691906s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513428296 (21h47m7.407136906s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513547533 (21h47m7.407256152s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513938307 (21h47m7.407646916s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514317599 (21h47m7.408026208s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514667566 (21h47m7.408376175s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:11.336233602 (21h47m8.229942212s) [upload=19126]: Next returned nil segment
Jan 16 13:43:11.336267347 (21h47m8.229975956s) [upload=19126]: Waiting for error group managing segments...
Jan 16 13:43:11.728156896 (21h47m8.621865516s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.744681339 (21h47m8.638389948s) [noise=true , address=81.171.14.70:4031 , piece_id=I2QFNTRNW5BSEW443K4HEMRPFWDWMS4A7OGLCBGL6MOBLBIVHXGA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.781770550 (21h47m8.675479159s) [noise=true , address=81.171.14.70:4039 , piece_id=PJJUDVG42DYTROSHW2EQ37F5KXADJDEJFE6WNHHZ756KVWQHOMMQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.060761834 (21h47m8.954470453s) [noise=true , address=185.167.204.205:4033 , piece_id=WYD6DRJKUXEGOYG2JKXTSPE6MGZ3PHAGTUKFAXXD5UZMOVDJPETQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066620951 (21h47m8.96032957s) [noise=true , address=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066990785 (21h47m8.960699404s) [noise=true , address=185.167.204.205:4031 , piece_id=7QWMBCQS2HLDT6LYENQCTPGQFCYSFHDBHZCEDCLTJ3XGYBEIEIUQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.067370788 (21h47m8.961079407s) [noise=true , address=185.167.204.205:4027 , piece_id=65HE6VZ67VQWRVZ52QTIOJR4YTZVEZHW4YXLTB3I3MMACX3QU5KQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.070378284 (21h47m8.964086903s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080751365 (21h47m8.974459984s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080916139 (21h47m8.974624748s) [noise=true , address=185.167.204.205:4023 , piece_id=SP3YCSO77HYXX3XHWPWZM4KZNESTVCVQN74T4X4AZHIMGSXHYCYQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.081172958 (21h47m8.974881567s) [noise=true , address=185.167.204.205:4019 , piece_id=MOY2FK2KGUGPPJ2IT5564P4O4Y72ELKITH43E76MQNK5WBF2ZI2A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.090478015 (21h47m8.984186624s) [noise=true , address=185.167.204.205:4013 , piece_id=CAJYKOAHLSKEAAM6BR3S4G24JB7CZYAQ2MSEJBRC4PYTPSTFZD5A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.095728263 (21h47m8.989436872s) [noise=true , address=185.167.204.206:4064 , piece_id=4MG2QWPMQD6BYDRJVCBPLMAYOECLVNG3YZJ6JPK6RDKXMWMXUEFA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.097262153 (21h47m8.990970763s) [noise=true , address=185.167.204.205:4034 , piece_id=GX5ZKXOND7HFY6QSA5H4BAOS343XACNZYU7CWY5LBY3TISMJBRBQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.099826616 (21h47m8.993535225s) [noise=true , address=185.167.204.206:4058 , piece_id=QLKSPOBUDDAPJK5R2EMJP74XM6TEBQLRSCBW2D4SUDYLAF5GAWZA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.111103288 (21h47m9.004811897s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.122116057 (21h47m9.015824666s) [noise=true , address=185.167.204.205:4002 , piece_id=IF42VMTKTFOZPV7MVZ6XEZA2VZQ4W7OB6GOU56AGM4V6UCICLKKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.126871291 (21h47m9.0205799s) [noise=true , address=185.167.204.205:4015 , piece_id=FILYFJPIFRGG7GWSPSWQY2IBXAJPFNOUWGCP234KANWQJD5XPZUA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.127756125 (21h47m9.021464734s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.131797441 (21h47m9.02550605s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.152285220 (21h47m9.045993839s) [noise=true , address=185.167.204.206:4054 , piece_id=OR36BJX2U7KWL3YGZDAMRNLC44IDHRD4FXVZDP4XDLVWA73YHCIA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.154173035 (21h47m9.047881644s) [noise=true , address=185.167.204.206:4057 , piece_id=WXMMXS66XDZLHNXKIR44R5Q54CLCQN77YMKOZ2RM7MPVTXT3M67A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.397697308 (21h47m9.291405927s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.847835054 (21h47m9.741543664s) [noise=true , address=89.149.201.147:4003 , piece_id=E72MB7PBJQUMJJLOEQC4FDZO6PEZW5S2I6UH46JDPQBS7CKGHTJQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:44:09.858617030 (21h48m6.752325639s) [noise=true , address=89.149.201.147:4001 , piece_id=RBW4Y52Y7K4UPKZ6J67HOW5VN75YUFIHFRUYZYE3Y2O6WPOTIYXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection
Jan 16 13:44:09.858698235 (21h48m6.752406864s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE, address:89.149.201.147:4022): protocol: read tcp 188.166.225.170:49424->89.149.201.147:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858727541 (21h48m6.75243616s) [noise=true , address=89.149.201.147:4017 , piece_id=VOZD426Q4XUTLC7WBYGSQ54BS6VK4KP5TQMZ7R4SNYDRZXBEIW6A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection
Jan 16 13:44:09.858851426 (21h48m6.752560035s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 188.166.225.170:59878->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858875492 (21h48m6.752584101s) [seg_pos={0 0}, upload=19126]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Jan 16 13:44:10.026477265 (21h48m6.920185874s) [upload=19126]: Done uploading segments...
Jan 16 13:44:10.026510469 (21h48m6.920219088s) [upload=19126]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.670533255s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
25.414150888s trace:
https://jaeger.dev.storj.io/trace/6b9a0c4bfefdaf17
FAIL (2026-01-16 13:45:12 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 25.414175585s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXXX] duration:
30.001181306s trace:
https://jaeger.dev.storj.io/trace/2e84880cc157dafc
FAIL (2026-01-16 13:45:09 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001211783s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [............................X.............X........X..X.XX..] duration:
4.786711894s trace:
https://jaeger.dev.storj.io/trace/287073b7bf0613bb
FAIL (2026-01-16 13:44:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
s=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512857569 (21h47m7.406566178s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512903297 (21h47m7.406611906s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513163702 (21h47m7.406872311s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513210451 (21h47m7.40691906s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513428296 (21h47m7.407136906s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513547533 (21h47m7.407256152s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513938307 (21h47m7.407646916s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514317599 (21h47m7.408026208s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514667566 (21h47m7.408376175s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:11.336233602 (21h47m8.229942212s) [upload=19126]: Next returned nil segment
Jan 16 13:43:11.336267347 (21h47m8.229975956s) [upload=19126]: Waiting for error group managing segments...
Jan 16 13:43:11.728156896 (21h47m8.621865516s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.744681339 (21h47m8.638389948s) [noise=true , address=81.171.14.70:4031 , piece_id=I2QFNTRNW5BSEW443K4HEMRPFWDWMS4A7OGLCBGL6MOBLBIVHXGA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.781770550 (21h47m8.675479159s) [noise=true , address=81.171.14.70:4039 , piece_id=PJJUDVG42DYTROSHW2EQ37F5KXADJDEJFE6WNHHZ756KVWQHOMMQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.060761834 (21h47m8.954470453s) [noise=true , address=185.167.204.205:4033 , piece_id=WYD6DRJKUXEGOYG2JKXTSPE6MGZ3PHAGTUKFAXXD5UZMOVDJPETQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066620951 (21h47m8.96032957s) [noise=true , address=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066990785 (21h47m8.960699404s) [noise=true , address=185.167.204.205:4031 , piece_id=7QWMBCQS2HLDT6LYENQCTPGQFCYSFHDBHZCEDCLTJ3XGYBEIEIUQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.067370788 (21h47m8.961079407s) [noise=true , address=185.167.204.205:4027 , piece_id=65HE6VZ67VQWRVZ52QTIOJR4YTZVEZHW4YXLTB3I3MMACX3QU5KQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.070378284 (21h47m8.964086903s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080751365 (21h47m8.974459984s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080916139 (21h47m8.974624748s) [noise=true , address=185.167.204.205:4023 , piece_id=SP3YCSO77HYXX3XHWPWZM4KZNESTVCVQN74T4X4AZHIMGSXHYCYQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.081172958 (21h47m8.974881567s) [noise=true , address=185.167.204.205:4019 , piece_id=MOY2FK2KGUGPPJ2IT5564P4O4Y72ELKITH43E76MQNK5WBF2ZI2A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.090478015 (21h47m8.984186624s) [noise=true , address=185.167.204.205:4013 , piece_id=CAJYKOAHLSKEAAM6BR3S4G24JB7CZYAQ2MSEJBRC4PYTPSTFZD5A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.095728263 (21h47m8.989436872s) [noise=true , address=185.167.204.206:4064 , piece_id=4MG2QWPMQD6BYDRJVCBPLMAYOECLVNG3YZJ6JPK6RDKXMWMXUEFA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.097262153 (21h47m8.990970763s) [noise=true , address=185.167.204.205:4034 , piece_id=GX5ZKXOND7HFY6QSA5H4BAOS343XACNZYU7CWY5LBY3TISMJBRBQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.099826616 (21h47m8.993535225s) [noise=true , address=185.167.204.206:4058 , piece_id=QLKSPOBUDDAPJK5R2EMJP74XM6TEBQLRSCBW2D4SUDYLAF5GAWZA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.111103288 (21h47m9.004811897s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.122116057 (21h47m9.015824666s) [noise=true , address=185.167.204.205:4002 , piece_id=IF42VMTKTFOZPV7MVZ6XEZA2VZQ4W7OB6GOU56AGM4V6UCICLKKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.126871291 (21h47m9.0205799s) [noise=true , address=185.167.204.205:4015 , piece_id=FILYFJPIFRGG7GWSPSWQY2IBXAJPFNOUWGCP234KANWQJD5XPZUA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.127756125 (21h47m9.021464734s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.131797441 (21h47m9.02550605s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.152285220 (21h47m9.045993839s) [noise=true , address=185.167.204.206:4054 , piece_id=OR36BJX2U7KWL3YGZDAMRNLC44IDHRD4FXVZDP4XDLVWA73YHCIA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.154173035 (21h47m9.047881644s) [noise=true , address=185.167.204.206:4057 , piece_id=WXMMXS66XDZLHNXKIR44R5Q54CLCQN77YMKOZ2RM7MPVTXT3M67A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.397697308 (21h47m9.291405927s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.847835054 (21h47m9.741543664s) [noise=true , address=89.149.201.147:4003 , piece_id=E72MB7PBJQUMJJLOEQC4FDZO6PEZW5S2I6UH46JDPQBS7CKGHTJQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:44:09.858617030 (21h48m6.752325639s) [noise=true , address=89.149.201.147:4001 , piece_id=RBW4Y52Y7K4UPKZ6J67HOW5VN75YUFIHFRUYZYE3Y2O6WPOTIYXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection
Jan 16 13:44:09.858698235 (21h48m6.752406864s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE, address:89.149.201.147:4022): protocol: read tcp 188.166.225.170:49424->89.149.201.147:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858727541 (21h48m6.75243616s) [noise=true , address=89.149.201.147:4017 , piece_id=VOZD426Q4XUTLC7WBYGSQ54BS6VK4KP5TQMZ7R4SNYDRZXBEIW6A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection
Jan 16 13:44:09.858851426 (21h48m6.752560035s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 188.166.225.170:59878->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858875492 (21h48m6.752584101s) [seg_pos={0 0}, upload=19126]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Jan 16 13:44:10.026477265 (21h48m6.920185874s) [upload=19126]: Done uploading segments...
Jan 16 13:44:10.026510469 (21h48m6.920219088s) [upload=19126]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.786741379s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [.X..XX..X.............X.X............X.............X..X..X..] duration:
34.578621536s trace:
https://jaeger.dev.storj.io/trace/5475a631601811f5
FAIL (2026-01-16 13:44:53 UTC)
orbiter warning: context deadline exceeded
Duration: 34.578651182s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000844143s trace:
https://jaeger.dev.storj.io/trace/669426b8dd677181
FAIL (2026-01-16 13:44:48 UTC)
orbiter warning: context deadline exceeded
Duration: 5.00086419s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
51 out of 60 is failed (threshold=3) [XXXXXXXXXX.XXXXXXXX.XXXX..XXX..XXX.X..XXXXXXXXXXXXXXXXXXXXXX] duration:
30.001625352s trace:
https://jaeger.dev.storj.io/trace/2c1eddc72c98f4f8
FAIL (2026-01-16 13:44:39 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001650729s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
60 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000996174s trace:
https://jaeger.dev.storj.io/trace/75163eefbdffc107
FAIL (2026-01-16 13:44:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001013236s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.............................X.............X........X..X.XX.] duration:
5.23837711s trace:
https://jaeger.dev.storj.io/trace/287073b7bf0613bb
FAIL (2026-01-16 13:44:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
s=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512857569 (21h47m7.406566178s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512903297 (21h47m7.406611906s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513163702 (21h47m7.406872311s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513210451 (21h47m7.40691906s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513428296 (21h47m7.407136906s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513547533 (21h47m7.407256152s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513938307 (21h47m7.407646916s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514317599 (21h47m7.408026208s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514667566 (21h47m7.408376175s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:11.336233602 (21h47m8.229942212s) [upload=19126]: Next returned nil segment
Jan 16 13:43:11.336267347 (21h47m8.229975956s) [upload=19126]: Waiting for error group managing segments...
Jan 16 13:43:11.728156896 (21h47m8.621865516s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.744681339 (21h47m8.638389948s) [noise=true , address=81.171.14.70:4031 , piece_id=I2QFNTRNW5BSEW443K4HEMRPFWDWMS4A7OGLCBGL6MOBLBIVHXGA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.781770550 (21h47m8.675479159s) [noise=true , address=81.171.14.70:4039 , piece_id=PJJUDVG42DYTROSHW2EQ37F5KXADJDEJFE6WNHHZ756KVWQHOMMQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.060761834 (21h47m8.954470453s) [noise=true , address=185.167.204.205:4033 , piece_id=WYD6DRJKUXEGOYG2JKXTSPE6MGZ3PHAGTUKFAXXD5UZMOVDJPETQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066620951 (21h47m8.96032957s) [noise=true , address=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066990785 (21h47m8.960699404s) [noise=true , address=185.167.204.205:4031 , piece_id=7QWMBCQS2HLDT6LYENQCTPGQFCYSFHDBHZCEDCLTJ3XGYBEIEIUQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.067370788 (21h47m8.961079407s) [noise=true , address=185.167.204.205:4027 , piece_id=65HE6VZ67VQWRVZ52QTIOJR4YTZVEZHW4YXLTB3I3MMACX3QU5KQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.070378284 (21h47m8.964086903s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080751365 (21h47m8.974459984s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080916139 (21h47m8.974624748s) [noise=true , address=185.167.204.205:4023 , piece_id=SP3YCSO77HYXX3XHWPWZM4KZNESTVCVQN74T4X4AZHIMGSXHYCYQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.081172958 (21h47m8.974881567s) [noise=true , address=185.167.204.205:4019 , piece_id=MOY2FK2KGUGPPJ2IT5564P4O4Y72ELKITH43E76MQNK5WBF2ZI2A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.090478015 (21h47m8.984186624s) [noise=true , address=185.167.204.205:4013 , piece_id=CAJYKOAHLSKEAAM6BR3S4G24JB7CZYAQ2MSEJBRC4PYTPSTFZD5A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.095728263 (21h47m8.989436872s) [noise=true , address=185.167.204.206:4064 , piece_id=4MG2QWPMQD6BYDRJVCBPLMAYOECLVNG3YZJ6JPK6RDKXMWMXUEFA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.097262153 (21h47m8.990970763s) [noise=true , address=185.167.204.205:4034 , piece_id=GX5ZKXOND7HFY6QSA5H4BAOS343XACNZYU7CWY5LBY3TISMJBRBQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.099826616 (21h47m8.993535225s) [noise=true , address=185.167.204.206:4058 , piece_id=QLKSPOBUDDAPJK5R2EMJP74XM6TEBQLRSCBW2D4SUDYLAF5GAWZA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.111103288 (21h47m9.004811897s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.122116057 (21h47m9.015824666s) [noise=true , address=185.167.204.205:4002 , piece_id=IF42VMTKTFOZPV7MVZ6XEZA2VZQ4W7OB6GOU56AGM4V6UCICLKKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.126871291 (21h47m9.0205799s) [noise=true , address=185.167.204.205:4015 , piece_id=FILYFJPIFRGG7GWSPSWQY2IBXAJPFNOUWGCP234KANWQJD5XPZUA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.127756125 (21h47m9.021464734s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.131797441 (21h47m9.02550605s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.152285220 (21h47m9.045993839s) [noise=true , address=185.167.204.206:4054 , piece_id=OR36BJX2U7KWL3YGZDAMRNLC44IDHRD4FXVZDP4XDLVWA73YHCIA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.154173035 (21h47m9.047881644s) [noise=true , address=185.167.204.206:4057 , piece_id=WXMMXS66XDZLHNXKIR44R5Q54CLCQN77YMKOZ2RM7MPVTXT3M67A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.397697308 (21h47m9.291405927s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.847835054 (21h47m9.741543664s) [noise=true , address=89.149.201.147:4003 , piece_id=E72MB7PBJQUMJJLOEQC4FDZO6PEZW5S2I6UH46JDPQBS7CKGHTJQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:44:09.858617030 (21h48m6.752325639s) [noise=true , address=89.149.201.147:4001 , piece_id=RBW4Y52Y7K4UPKZ6J67HOW5VN75YUFIHFRUYZYE3Y2O6WPOTIYXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection
Jan 16 13:44:09.858698235 (21h48m6.752406864s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE, address:89.149.201.147:4022): protocol: read tcp 188.166.225.170:49424->89.149.201.147:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858727541 (21h48m6.75243616s) [noise=true , address=89.149.201.147:4017 , piece_id=VOZD426Q4XUTLC7WBYGSQ54BS6VK4KP5TQMZ7R4SNYDRZXBEIW6A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection
Jan 16 13:44:09.858851426 (21h48m6.752560035s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 188.166.225.170:59878->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858875492 (21h48m6.752584101s) [seg_pos={0 0}, upload=19126]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Jan 16 13:44:10.026477265 (21h48m6.920185874s) [upload=19126]: Done uploading segments...
Jan 16 13:44:10.026510469 (21h48m6.920219088s) [upload=19126]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 5.238392699s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..............................X.............X........X..X.XX] duration:
1m0.170667311s trace:
https://jaeger.dev.storj.io/trace/287073b7bf0613bb
FAIL (2026-01-16 13:44:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
s=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512857569 (21h47m7.406566178s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.512903297 (21h47m7.406611906s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513163702 (21h47m7.406872311s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513210451 (21h47m7.40691906s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513428296 (21h47m7.407136906s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513547533 (21h47m7.407256152s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.513938307 (21h47m7.407646916s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514317599 (21h47m7.408026208s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:10.514667566 (21h47m7.408376175s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Uploading piece...
Jan 16 13:43:11.336233602 (21h47m8.229942212s) [upload=19126]: Next returned nil segment
Jan 16 13:43:11.336267347 (21h47m8.229975956s) [upload=19126]: Waiting for error group managing segments...
Jan 16 13:43:11.728156896 (21h47m8.621865516s) [noise=true , address=81.171.14.70:4030 , piece_id=QIICEONDSHSCEMSRFAVH4XKXK3EOUPPRFWGGJD5RB3JIMGE7JRLQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.744681339 (21h47m8.638389948s) [noise=true , address=81.171.14.70:4031 , piece_id=I2QFNTRNW5BSEW443K4HEMRPFWDWMS4A7OGLCBGL6MOBLBIVHXGA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:11.781770550 (21h47m8.675479159s) [noise=true , address=81.171.14.70:4039 , piece_id=PJJUDVG42DYTROSHW2EQ37F5KXADJDEJFE6WNHHZ756KVWQHOMMQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.060761834 (21h47m8.954470453s) [noise=true , address=185.167.204.205:4033 , piece_id=WYD6DRJKUXEGOYG2JKXTSPE6MGZ3PHAGTUKFAXXD5UZMOVDJPETQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066620951 (21h47m8.96032957s) [noise=true , address=185.167.204.205:4026 , piece_id=AIWA5MU2M7I5DLQJXA7UU7KBTGA4BU7EA3QIVWA45AEXIE6CZSCA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.066990785 (21h47m8.960699404s) [noise=true , address=185.167.204.205:4031 , piece_id=7QWMBCQS2HLDT6LYENQCTPGQFCYSFHDBHZCEDCLTJ3XGYBEIEIUQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.067370788 (21h47m8.961079407s) [noise=true , address=185.167.204.205:4027 , piece_id=65HE6VZ67VQWRVZ52QTIOJR4YTZVEZHW4YXLTB3I3MMACX3QU5KQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.070378284 (21h47m8.964086903s) [noise=true , address=185.167.204.206:4051 , piece_id=Z43RVXK5I6WZLKADZ22KVOFENVIUXMQYR7PB56BTXSW4UJ3L25ZQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080751365 (21h47m8.974459984s) [noise=true , address=185.167.204.205:4001 , piece_id=UZ2AG2JPLZ4Q326UYYZZ773DKYPMUIA6EHG54TMFGY3JM6HM4FXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.080916139 (21h47m8.974624748s) [noise=true , address=185.167.204.205:4023 , piece_id=SP3YCSO77HYXX3XHWPWZM4KZNESTVCVQN74T4X4AZHIMGSXHYCYQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.081172958 (21h47m8.974881567s) [noise=true , address=185.167.204.205:4019 , piece_id=MOY2FK2KGUGPPJ2IT5564P4O4Y72ELKITH43E76MQNK5WBF2ZI2A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.090478015 (21h47m8.984186624s) [noise=true , address=185.167.204.205:4013 , piece_id=CAJYKOAHLSKEAAM6BR3S4G24JB7CZYAQ2MSEJBRC4PYTPSTFZD5A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.095728263 (21h47m8.989436872s) [noise=true , address=185.167.204.206:4064 , piece_id=4MG2QWPMQD6BYDRJVCBPLMAYOECLVNG3YZJ6JPK6RDKXMWMXUEFA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.097262153 (21h47m8.990970763s) [noise=true , address=185.167.204.205:4034 , piece_id=GX5ZKXOND7HFY6QSA5H4BAOS343XACNZYU7CWY5LBY3TISMJBRBQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.099826616 (21h47m8.993535225s) [noise=true , address=185.167.204.206:4058 , piece_id=QLKSPOBUDDAPJK5R2EMJP74XM6TEBQLRSCBW2D4SUDYLAF5GAWZA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.111103288 (21h47m9.004811897s) [noise=true , address=185.167.204.205:4020 , piece_id=NECEB5MJ53GJZ2EDNOEZFCVXDVAYY2BOK2YTHCEMFLTWYQWLZF3A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.122116057 (21h47m9.015824666s) [noise=true , address=185.167.204.205:4002 , piece_id=IF42VMTKTFOZPV7MVZ6XEZA2VZQ4W7OB6GOU56AGM4V6UCICLKKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.126871291 (21h47m9.0205799s) [noise=true , address=185.167.204.205:4015 , piece_id=FILYFJPIFRGG7GWSPSWQY2IBXAJPFNOUWGCP234KANWQJD5XPZUA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.127756125 (21h47m9.021464734s) [noise=true , address=185.167.204.206:4050 , piece_id=UCYS2IFO3GP4RSIHQR3Z2JNZIA4VOQ6UJ7HT2VUIDJQCMTDISCKA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.131797441 (21h47m9.02550605s) [noise=true , address=185.167.204.206:4048 , piece_id=RV4N243X4LDIL7TM7QWT5HC2OTYI5Y6GNU4JSFMNEA2FBDUTM5RQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.152285220 (21h47m9.045993839s) [noise=true , address=185.167.204.206:4054 , piece_id=OR36BJX2U7KWL3YGZDAMRNLC44IDHRD4FXVZDP4XDLVWA73YHCIA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.154173035 (21h47m9.047881644s) [noise=true , address=185.167.204.206:4057 , piece_id=WXMMXS66XDZLHNXKIR44R5Q54CLCQN77YMKOZ2RM7MPVTXT3M67A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.397697308 (21h47m9.291405927s) [noise=true , address=89.149.201.147:4023 , piece_id=JXESVMEYDTWWMYZ6JEDOQYYX5KQ4DJH7XLEXF5HB25QBPKSGBWTQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:43:12.847835054 (21h47m9.741543664s) [noise=true , address=89.149.201.147:4003 , piece_id=E72MB7PBJQUMJJLOEQC4FDZO6PEZW5S2I6UH46JDPQBS7CKGHTJQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: <nil>
Jan 16 13:44:09.858617030 (21h48m6.752325639s) [noise=true , address=89.149.201.147:4001 , piece_id=RBW4Y52Y7K4UPKZ6J67HOW5VN75YUFIHFRUYZYE3Y2O6WPOTIYXA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40250->89.149.201.147:4001: use of closed network connection
Jan 16 13:44:09.858698235 (21h48m6.752406864s) [noise=true , address=89.149.201.147:4022 , piece_id=6S5IHAI447N64RYXL2C6HRTQOUOPXST2DSVCWN53OMT7ZA3CIFXQ, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE, address:89.149.201.147:4022): protocol: read tcp 188.166.225.170:49424->89.149.201.147:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858727541 (21h48m6.75243616s) [noise=true , address=89.149.201.147:4017 , piece_id=VOZD426Q4XUTLC7WBYGSQ54BS6VK4KP5TQMZ7R4SNYDRZXBEIW6A, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40586->89.149.201.147:4017: use of closed network connection
Jan 16 13:44:09.858851426 (21h48m6.752560035s) [noise=true , address=89.149.201.147:4005 , piece_id=K2PZNU2J42JAGPFZMRZFIKXN4UUXYKLGEKTDUWT2XWCJCVXEUQNA, seg_pos={0 0}, upload=19126]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 188.166.225.170:59878->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 16 13:44:09.858875492 (21h48m6.752584101s) [seg_pos={0 0}, upload=19126]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Jan 16 13:44:10.026477265 (21h48m6.920185874s) [upload=19126]: Done uploading segments...
Jan 16 13:44:10.026510469 (21h48m6.920219088s) [upload=19126]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 1m0.170691237s
core
uplink
eu-select-1
eu1-placement-15
|