Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.809821998s trace:
https://jaeger.dev.storj.io/trace/1a859d3df70f9e86
FAIL (2025-12-09 04:08:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259131101 (84h33m3.130214146s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259596488 (84h33m3.130679523s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259977563 (84h33m3.131060609s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260318812 (84h33m3.131401858s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260685692 (84h33m3.131768727s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261095021 (84h33m3.132178066s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261440168 (84h33m3.132523214s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261787410 (84h33m3.132870455s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262212559 (84h33m3.133295604s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262561884 (84h33m3.133644929s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:04.083723384 (84h33m3.954806419s) [upload=73832]: Next returned nil segment
Dec 9 04:02:04.083741359 (84h33m3.954824404s) [upload=73832]: Waiting for error group managing segments...
Dec 9 04:02:04.485138566 (84h33m4.356221611s) [noise=true , address=185.167.204.206:4055 , piece_id=UGHZZA675KF62AO4TDKVZKMBN76NB2KB73Q4CHO6K5XJPEOSNHBQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.485953738 (84h33m4.357036783s) [noise=true , address=185.167.204.205:4033 , piece_id=UD4WVCCYPYUNOF73JBNLU4GGN7Q5JYSSTHRYWL6WWBWH7REFWDPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.486662538 (84h33m4.357745583s) [noise=true , address=185.167.204.206:4063 , piece_id=AEXLZ373NNMQVLLFNDWCKMKWZ6VD5VL6GDKMEJQIVQFH4LFVDG7A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487276066 (84h33m4.358359111s) [noise=true , address=185.167.204.206:4067 , piece_id=DRQW74HZAECN5ZXX5Z6ZDB6XSV7YDFFTEBUZA3IVGC4AZBOVM7QQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487951051 (84h33m4.359034096s) [noise=true , address=185.167.204.206:4056 , piece_id=UNGF5SQQR6T7ZSF2TC6HQCY46Z7IDHQ2Z2W554WIXFWBQ2KZMXPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.488930937 (84h33m4.360013982s) [noise=true , address=185.167.204.205:4036 , piece_id=KFD6YVTYN3ZORNZISSZWULGTM5HQM3XKMSZUEA2JRH647VP5PA6A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.497593282 (84h33m4.368676327s) [noise=true , address=185.167.204.205:4006 , piece_id=BH5W3QAJQLULFE3BWIGHDASLW7BFPXCBFNSEEHSLXNHK6HVS42TA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.499663965 (84h33m4.37074701s) [noise=true , address=185.167.204.206:4050 , piece_id=R2NZYKV2XN43HPN35Z2RCDMRHINYTGX3MISTXFG7QWZUS326D2FQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.500078284 (84h33m4.371161319s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.503424576 (84h33m4.374507621s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.504155277 (84h33m4.375238332s) [noise=true , address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.506781639 (84h33m4.377864694s) [noise=true , address=185.167.204.205:4005 , piece_id=VY7HILMJLYIMOF7XQ5JI2BYG3HL6CSC344FEX6526LEPLRAYNZXA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.507557626 (84h33m4.378640671s) [noise=true , address=185.167.204.206:4065 , piece_id=EK2QQCWVYSEYEB6TQUMRWIJL5QNKOCBNANOW5BMHP5TZMMILTK2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.510750356 (84h33m4.381833401s) [noise=true , address=185.167.204.205:4028 , piece_id=O5YHC7OYSPVPFE6L4ME3QYLON4WOAH5AYHV2KRW3FYEYQLJVIYSQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.524221958 (84h33m4.395305003s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.064989944 (84h33m4.936072989s) [noise=true , address=81.171.14.70:4043 , piece_id=3P3XPFAETWIVCJRB2UU5SZWURCVOGVEC3VXFA6FX4QMWQHLILTUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.087256625 (84h33m4.95833967s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.103625815 (84h33m4.97470886s) [noise=true , address=81.171.14.70:4027 , piece_id=2MVY2IXUDMNSLKZ4A2WEYKAOAOPY5IV354FFFICUCIHVKAOFL5HQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.110184184 (84h33m4.981267229s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.536877270 (84h33m5.407960315s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.545643723 (84h33m5.416726768s) [noise=true , address=81.171.14.70:4048 , piece_id=PLTYOGFFQAXYY725S6HXLR3COY26JZYHMAJMCKJSOK5FGFDUW5DA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.547511790 (84h33m5.418594835s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.549473886 (84h33m5.420556931s) [noise=true , address=81.171.14.70:4045 , piece_id=6GCAH2P7YMZR6E6KFQ4I3BKUCKW4XNNNRGZMKRQ46DRGHCB57R4A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.649216977 (84h33m5.520300022s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.736540175 (84h33m5.607623221s) [noise=true , address=89.149.201.147:4019 , piece_id=GM3THIX26QZBTAVKNSSNGF4MWUKH2NHN6QKQTAZR7I5AS7V55ZDQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739705292 (84h33m5.610788338s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739730531 (84h33m5.610813576s) [seg_pos={0 0}, upload=73832]: Segment reached optimal threshold of 26 pieces.
Dec 9 04:02:05.739992771 (84h33m5.611075836s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740003922 (84h33m5.611086967s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740059076 (84h33m5.611142121s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740088111 (84h33m5.611171157s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740123359 (84h33m5.611206404s) [seg_pos={0 0}, upload=73832]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 04:02:05.740217909 (84h33m5.611300944s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 04:02:05.740225062 (84h33m5.611308107s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 04:05:41.230845175 (84h36m41.10192822s) [upload=73832]: Done uploading segments...
Dec 9 04:05:41.230955645 (84h36m41.102038691s) [upload=73832]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.809849731s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [......X..XX....X.........X...................XX.............] duration:
5.671650613s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:08:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.671675101s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.X............X.......................X...............X.....] duration:
3.518714899s trace:
https://jaeger.dev.storj.io/trace/8c711ae8f1c72a8
FAIL (2025-12-09 04:07:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.518740638s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...................X......X.........................X.......] duration:
4.917756816s trace:
https://jaeger.dev.storj.io/trace/327471f8d037add8
FAIL (2025-12-09 04:07:43 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.917856306s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
5.702011459s trace:
https://jaeger.dev.storj.io/trace/1a859d3df70f9e86
FAIL (2025-12-09 04:07:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259131101 (84h33m3.130214146s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259596488 (84h33m3.130679523s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259977563 (84h33m3.131060609s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260318812 (84h33m3.131401858s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260685692 (84h33m3.131768727s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261095021 (84h33m3.132178066s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261440168 (84h33m3.132523214s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261787410 (84h33m3.132870455s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262212559 (84h33m3.133295604s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262561884 (84h33m3.133644929s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:04.083723384 (84h33m3.954806419s) [upload=73832]: Next returned nil segment
Dec 9 04:02:04.083741359 (84h33m3.954824404s) [upload=73832]: Waiting for error group managing segments...
Dec 9 04:02:04.485138566 (84h33m4.356221611s) [noise=true , address=185.167.204.206:4055 , piece_id=UGHZZA675KF62AO4TDKVZKMBN76NB2KB73Q4CHO6K5XJPEOSNHBQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.485953738 (84h33m4.357036783s) [noise=true , address=185.167.204.205:4033 , piece_id=UD4WVCCYPYUNOF73JBNLU4GGN7Q5JYSSTHRYWL6WWBWH7REFWDPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.486662538 (84h33m4.357745583s) [noise=true , address=185.167.204.206:4063 , piece_id=AEXLZ373NNMQVLLFNDWCKMKWZ6VD5VL6GDKMEJQIVQFH4LFVDG7A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487276066 (84h33m4.358359111s) [noise=true , address=185.167.204.206:4067 , piece_id=DRQW74HZAECN5ZXX5Z6ZDB6XSV7YDFFTEBUZA3IVGC4AZBOVM7QQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487951051 (84h33m4.359034096s) [noise=true , address=185.167.204.206:4056 , piece_id=UNGF5SQQR6T7ZSF2TC6HQCY46Z7IDHQ2Z2W554WIXFWBQ2KZMXPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.488930937 (84h33m4.360013982s) [noise=true , address=185.167.204.205:4036 , piece_id=KFD6YVTYN3ZORNZISSZWULGTM5HQM3XKMSZUEA2JRH647VP5PA6A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.497593282 (84h33m4.368676327s) [noise=true , address=185.167.204.205:4006 , piece_id=BH5W3QAJQLULFE3BWIGHDASLW7BFPXCBFNSEEHSLXNHK6HVS42TA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.499663965 (84h33m4.37074701s) [noise=true , address=185.167.204.206:4050 , piece_id=R2NZYKV2XN43HPN35Z2RCDMRHINYTGX3MISTXFG7QWZUS326D2FQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.500078284 (84h33m4.371161319s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.503424576 (84h33m4.374507621s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.504155277 (84h33m4.375238332s) [noise=true , address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.506781639 (84h33m4.377864694s) [noise=true , address=185.167.204.205:4005 , piece_id=VY7HILMJLYIMOF7XQ5JI2BYG3HL6CSC344FEX6526LEPLRAYNZXA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.507557626 (84h33m4.378640671s) [noise=true , address=185.167.204.206:4065 , piece_id=EK2QQCWVYSEYEB6TQUMRWIJL5QNKOCBNANOW5BMHP5TZMMILTK2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.510750356 (84h33m4.381833401s) [noise=true , address=185.167.204.205:4028 , piece_id=O5YHC7OYSPVPFE6L4ME3QYLON4WOAH5AYHV2KRW3FYEYQLJVIYSQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.524221958 (84h33m4.395305003s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.064989944 (84h33m4.936072989s) [noise=true , address=81.171.14.70:4043 , piece_id=3P3XPFAETWIVCJRB2UU5SZWURCVOGVEC3VXFA6FX4QMWQHLILTUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.087256625 (84h33m4.95833967s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.103625815 (84h33m4.97470886s) [noise=true , address=81.171.14.70:4027 , piece_id=2MVY2IXUDMNSLKZ4A2WEYKAOAOPY5IV354FFFICUCIHVKAOFL5HQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.110184184 (84h33m4.981267229s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.536877270 (84h33m5.407960315s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.545643723 (84h33m5.416726768s) [noise=true , address=81.171.14.70:4048 , piece_id=PLTYOGFFQAXYY725S6HXLR3COY26JZYHMAJMCKJSOK5FGFDUW5DA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.547511790 (84h33m5.418594835s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.549473886 (84h33m5.420556931s) [noise=true , address=81.171.14.70:4045 , piece_id=6GCAH2P7YMZR6E6KFQ4I3BKUCKW4XNNNRGZMKRQ46DRGHCB57R4A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.649216977 (84h33m5.520300022s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.736540175 (84h33m5.607623221s) [noise=true , address=89.149.201.147:4019 , piece_id=GM3THIX26QZBTAVKNSSNGF4MWUKH2NHN6QKQTAZR7I5AS7V55ZDQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739705292 (84h33m5.610788338s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739730531 (84h33m5.610813576s) [seg_pos={0 0}, upload=73832]: Segment reached optimal threshold of 26 pieces.
Dec 9 04:02:05.739992771 (84h33m5.611075836s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740003922 (84h33m5.611086967s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740059076 (84h33m5.611142121s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740088111 (84h33m5.611171157s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740123359 (84h33m5.611206404s) [seg_pos={0 0}, upload=73832]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 04:02:05.740217909 (84h33m5.611300944s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 04:02:05.740225062 (84h33m5.611308107s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 04:05:41.230845175 (84h36m41.10192822s) [upload=73832]: Done uploading segments...
Dec 9 04:05:41.230955645 (84h36m41.102038691s) [upload=73832]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.702037939s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..X............X.......................X...............X....] duration:
3.547061676s trace:
https://jaeger.dev.storj.io/trace/8c711ae8f1c72a8
FAIL (2025-12-09 04:06:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.547092164s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [.......X..XX....X.........X...................XX............] duration:
5.650860629s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:06:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.650886097s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....................X......X.........................X......] duration:
5.105363805s trace:
https://jaeger.dev.storj.io/trace/327471f8d037add8
FAIL (2025-12-09 04:06:46 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.105393932s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [........X..XX....X.........X...................XX...........] duration:
5.737091236s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:06:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.737119089s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.79169954s trace:
https://jaeger.dev.storj.io/trace/1a859d3df70f9e86
FAIL (2025-12-09 04:06:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259131101 (84h33m3.130214146s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259596488 (84h33m3.130679523s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259977563 (84h33m3.131060609s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260318812 (84h33m3.131401858s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260685692 (84h33m3.131768727s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261095021 (84h33m3.132178066s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261440168 (84h33m3.132523214s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261787410 (84h33m3.132870455s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262212559 (84h33m3.133295604s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262561884 (84h33m3.133644929s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:04.083723384 (84h33m3.954806419s) [upload=73832]: Next returned nil segment
Dec 9 04:02:04.083741359 (84h33m3.954824404s) [upload=73832]: Waiting for error group managing segments...
Dec 9 04:02:04.485138566 (84h33m4.356221611s) [noise=true , address=185.167.204.206:4055 , piece_id=UGHZZA675KF62AO4TDKVZKMBN76NB2KB73Q4CHO6K5XJPEOSNHBQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.485953738 (84h33m4.357036783s) [noise=true , address=185.167.204.205:4033 , piece_id=UD4WVCCYPYUNOF73JBNLU4GGN7Q5JYSSTHRYWL6WWBWH7REFWDPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.486662538 (84h33m4.357745583s) [noise=true , address=185.167.204.206:4063 , piece_id=AEXLZ373NNMQVLLFNDWCKMKWZ6VD5VL6GDKMEJQIVQFH4LFVDG7A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487276066 (84h33m4.358359111s) [noise=true , address=185.167.204.206:4067 , piece_id=DRQW74HZAECN5ZXX5Z6ZDB6XSV7YDFFTEBUZA3IVGC4AZBOVM7QQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487951051 (84h33m4.359034096s) [noise=true , address=185.167.204.206:4056 , piece_id=UNGF5SQQR6T7ZSF2TC6HQCY46Z7IDHQ2Z2W554WIXFWBQ2KZMXPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.488930937 (84h33m4.360013982s) [noise=true , address=185.167.204.205:4036 , piece_id=KFD6YVTYN3ZORNZISSZWULGTM5HQM3XKMSZUEA2JRH647VP5PA6A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.497593282 (84h33m4.368676327s) [noise=true , address=185.167.204.205:4006 , piece_id=BH5W3QAJQLULFE3BWIGHDASLW7BFPXCBFNSEEHSLXNHK6HVS42TA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.499663965 (84h33m4.37074701s) [noise=true , address=185.167.204.206:4050 , piece_id=R2NZYKV2XN43HPN35Z2RCDMRHINYTGX3MISTXFG7QWZUS326D2FQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.500078284 (84h33m4.371161319s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.503424576 (84h33m4.374507621s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.504155277 (84h33m4.375238332s) [noise=true , address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.506781639 (84h33m4.377864694s) [noise=true , address=185.167.204.205:4005 , piece_id=VY7HILMJLYIMOF7XQ5JI2BYG3HL6CSC344FEX6526LEPLRAYNZXA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.507557626 (84h33m4.378640671s) [noise=true , address=185.167.204.206:4065 , piece_id=EK2QQCWVYSEYEB6TQUMRWIJL5QNKOCBNANOW5BMHP5TZMMILTK2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.510750356 (84h33m4.381833401s) [noise=true , address=185.167.204.205:4028 , piece_id=O5YHC7OYSPVPFE6L4ME3QYLON4WOAH5AYHV2KRW3FYEYQLJVIYSQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.524221958 (84h33m4.395305003s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.064989944 (84h33m4.936072989s) [noise=true , address=81.171.14.70:4043 , piece_id=3P3XPFAETWIVCJRB2UU5SZWURCVOGVEC3VXFA6FX4QMWQHLILTUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.087256625 (84h33m4.95833967s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.103625815 (84h33m4.97470886s) [noise=true , address=81.171.14.70:4027 , piece_id=2MVY2IXUDMNSLKZ4A2WEYKAOAOPY5IV354FFFICUCIHVKAOFL5HQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.110184184 (84h33m4.981267229s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.536877270 (84h33m5.407960315s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.545643723 (84h33m5.416726768s) [noise=true , address=81.171.14.70:4048 , piece_id=PLTYOGFFQAXYY725S6HXLR3COY26JZYHMAJMCKJSOK5FGFDUW5DA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.547511790 (84h33m5.418594835s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.549473886 (84h33m5.420556931s) [noise=true , address=81.171.14.70:4045 , piece_id=6GCAH2P7YMZR6E6KFQ4I3BKUCKW4XNNNRGZMKRQ46DRGHCB57R4A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.649216977 (84h33m5.520300022s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.736540175 (84h33m5.607623221s) [noise=true , address=89.149.201.147:4019 , piece_id=GM3THIX26QZBTAVKNSSNGF4MWUKH2NHN6QKQTAZR7I5AS7V55ZDQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739705292 (84h33m5.610788338s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739730531 (84h33m5.610813576s) [seg_pos={0 0}, upload=73832]: Segment reached optimal threshold of 26 pieces.
Dec 9 04:02:05.739992771 (84h33m5.611075836s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740003922 (84h33m5.611086967s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740059076 (84h33m5.611142121s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740088111 (84h33m5.611171157s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740123359 (84h33m5.611206404s) [seg_pos={0 0}, upload=73832]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 04:02:05.740217909 (84h33m5.611300944s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 04:02:05.740225062 (84h33m5.611308107s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 04:05:41.230845175 (84h36m41.10192822s) [upload=73832]: Done uploading segments...
Dec 9 04:05:41.230955645 (84h36m41.102038691s) [upload=73832]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.79179916s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...X............X.......................X...............X...] duration:
3.721727385s trace:
https://jaeger.dev.storj.io/trace/8c711ae8f1c72a8
FAIL (2025-12-09 04:06:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.721759445s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
5.205919451s trace:
https://jaeger.dev.storj.io/trace/1a859d3df70f9e86
FAIL (2025-12-09 04:05:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259131101 (84h33m3.130214146s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259596488 (84h33m3.130679523s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259977563 (84h33m3.131060609s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260318812 (84h33m3.131401858s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260685692 (84h33m3.131768727s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261095021 (84h33m3.132178066s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261440168 (84h33m3.132523214s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261787410 (84h33m3.132870455s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262212559 (84h33m3.133295604s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262561884 (84h33m3.133644929s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:04.083723384 (84h33m3.954806419s) [upload=73832]: Next returned nil segment
Dec 9 04:02:04.083741359 (84h33m3.954824404s) [upload=73832]: Waiting for error group managing segments...
Dec 9 04:02:04.485138566 (84h33m4.356221611s) [noise=true , address=185.167.204.206:4055 , piece_id=UGHZZA675KF62AO4TDKVZKMBN76NB2KB73Q4CHO6K5XJPEOSNHBQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.485953738 (84h33m4.357036783s) [noise=true , address=185.167.204.205:4033 , piece_id=UD4WVCCYPYUNOF73JBNLU4GGN7Q5JYSSTHRYWL6WWBWH7REFWDPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.486662538 (84h33m4.357745583s) [noise=true , address=185.167.204.206:4063 , piece_id=AEXLZ373NNMQVLLFNDWCKMKWZ6VD5VL6GDKMEJQIVQFH4LFVDG7A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487276066 (84h33m4.358359111s) [noise=true , address=185.167.204.206:4067 , piece_id=DRQW74HZAECN5ZXX5Z6ZDB6XSV7YDFFTEBUZA3IVGC4AZBOVM7QQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487951051 (84h33m4.359034096s) [noise=true , address=185.167.204.206:4056 , piece_id=UNGF5SQQR6T7ZSF2TC6HQCY46Z7IDHQ2Z2W554WIXFWBQ2KZMXPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.488930937 (84h33m4.360013982s) [noise=true , address=185.167.204.205:4036 , piece_id=KFD6YVTYN3ZORNZISSZWULGTM5HQM3XKMSZUEA2JRH647VP5PA6A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.497593282 (84h33m4.368676327s) [noise=true , address=185.167.204.205:4006 , piece_id=BH5W3QAJQLULFE3BWIGHDASLW7BFPXCBFNSEEHSLXNHK6HVS42TA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.499663965 (84h33m4.37074701s) [noise=true , address=185.167.204.206:4050 , piece_id=R2NZYKV2XN43HPN35Z2RCDMRHINYTGX3MISTXFG7QWZUS326D2FQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.500078284 (84h33m4.371161319s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.503424576 (84h33m4.374507621s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.504155277 (84h33m4.375238332s) [noise=true , address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.506781639 (84h33m4.377864694s) [noise=true , address=185.167.204.205:4005 , piece_id=VY7HILMJLYIMOF7XQ5JI2BYG3HL6CSC344FEX6526LEPLRAYNZXA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.507557626 (84h33m4.378640671s) [noise=true , address=185.167.204.206:4065 , piece_id=EK2QQCWVYSEYEB6TQUMRWIJL5QNKOCBNANOW5BMHP5TZMMILTK2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.510750356 (84h33m4.381833401s) [noise=true , address=185.167.204.205:4028 , piece_id=O5YHC7OYSPVPFE6L4ME3QYLON4WOAH5AYHV2KRW3FYEYQLJVIYSQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.524221958 (84h33m4.395305003s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.064989944 (84h33m4.936072989s) [noise=true , address=81.171.14.70:4043 , piece_id=3P3XPFAETWIVCJRB2UU5SZWURCVOGVEC3VXFA6FX4QMWQHLILTUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.087256625 (84h33m4.95833967s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.103625815 (84h33m4.97470886s) [noise=true , address=81.171.14.70:4027 , piece_id=2MVY2IXUDMNSLKZ4A2WEYKAOAOPY5IV354FFFICUCIHVKAOFL5HQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.110184184 (84h33m4.981267229s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.536877270 (84h33m5.407960315s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.545643723 (84h33m5.416726768s) [noise=true , address=81.171.14.70:4048 , piece_id=PLTYOGFFQAXYY725S6HXLR3COY26JZYHMAJMCKJSOK5FGFDUW5DA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.547511790 (84h33m5.418594835s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.549473886 (84h33m5.420556931s) [noise=true , address=81.171.14.70:4045 , piece_id=6GCAH2P7YMZR6E6KFQ4I3BKUCKW4XNNNRGZMKRQ46DRGHCB57R4A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.649216977 (84h33m5.520300022s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.736540175 (84h33m5.607623221s) [noise=true , address=89.149.201.147:4019 , piece_id=GM3THIX26QZBTAVKNSSNGF4MWUKH2NHN6QKQTAZR7I5AS7V55ZDQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739705292 (84h33m5.610788338s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739730531 (84h33m5.610813576s) [seg_pos={0 0}, upload=73832]: Segment reached optimal threshold of 26 pieces.
Dec 9 04:02:05.739992771 (84h33m5.611075836s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740003922 (84h33m5.611086967s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740059076 (84h33m5.611142121s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740088111 (84h33m5.611171157s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740123359 (84h33m5.611206404s) [seg_pos={0 0}, upload=73832]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 04:02:05.740217909 (84h33m5.611300944s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 04:02:05.740225062 (84h33m5.611308107s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 04:05:41.230845175 (84h36m41.10192822s) [upload=73832]: Done uploading segments...
Dec 9 04:05:41.230955645 (84h36m41.102038691s) [upload=73832]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.205936993s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
3m40.781209155s trace:
https://jaeger.dev.storj.io/trace/1a859d3df70f9e86
FAIL (2025-12-09 04:05:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259131101 (84h33m3.130214146s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259596488 (84h33m3.130679523s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.259977563 (84h33m3.131060609s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260318812 (84h33m3.131401858s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.260685692 (84h33m3.131768727s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261095021 (84h33m3.132178066s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261440168 (84h33m3.132523214s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.261787410 (84h33m3.132870455s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262212559 (84h33m3.133295604s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:03.262561884 (84h33m3.133644929s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Uploading piece...
Dec 9 04:02:04.083723384 (84h33m3.954806419s) [upload=73832]: Next returned nil segment
Dec 9 04:02:04.083741359 (84h33m3.954824404s) [upload=73832]: Waiting for error group managing segments...
Dec 9 04:02:04.485138566 (84h33m4.356221611s) [noise=true , address=185.167.204.206:4055 , piece_id=UGHZZA675KF62AO4TDKVZKMBN76NB2KB73Q4CHO6K5XJPEOSNHBQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.485953738 (84h33m4.357036783s) [noise=true , address=185.167.204.205:4033 , piece_id=UD4WVCCYPYUNOF73JBNLU4GGN7Q5JYSSTHRYWL6WWBWH7REFWDPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.486662538 (84h33m4.357745583s) [noise=true , address=185.167.204.206:4063 , piece_id=AEXLZ373NNMQVLLFNDWCKMKWZ6VD5VL6GDKMEJQIVQFH4LFVDG7A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487276066 (84h33m4.358359111s) [noise=true , address=185.167.204.206:4067 , piece_id=DRQW74HZAECN5ZXX5Z6ZDB6XSV7YDFFTEBUZA3IVGC4AZBOVM7QQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.487951051 (84h33m4.359034096s) [noise=true , address=185.167.204.206:4056 , piece_id=UNGF5SQQR6T7ZSF2TC6HQCY46Z7IDHQ2Z2W554WIXFWBQ2KZMXPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.488930937 (84h33m4.360013982s) [noise=true , address=185.167.204.205:4036 , piece_id=KFD6YVTYN3ZORNZISSZWULGTM5HQM3XKMSZUEA2JRH647VP5PA6A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.497593282 (84h33m4.368676327s) [noise=true , address=185.167.204.205:4006 , piece_id=BH5W3QAJQLULFE3BWIGHDASLW7BFPXCBFNSEEHSLXNHK6HVS42TA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.499663965 (84h33m4.37074701s) [noise=true , address=185.167.204.206:4050 , piece_id=R2NZYKV2XN43HPN35Z2RCDMRHINYTGX3MISTXFG7QWZUS326D2FQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.500078284 (84h33m4.371161319s) [noise=true , address=185.167.204.206:4047 , piece_id=PZIVI6VDUAMO4DT6L3MUJQY3HTUKTRS2UZIC4GKSXPLV3GRS6FRQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.503424576 (84h33m4.374507621s) [noise=true , address=185.167.204.206:4059 , piece_id=E75T5HKLDLBWDWJY27QWINAHPARSEF7PHGPQVOLKBXFMGXTMSL2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.504155277 (84h33m4.375238332s) [noise=true , address=185.167.204.205:4025 , piece_id=LECBZNBR4MHNZWNOZFAXHVBNKVAM3APFCW3OTZLKPEMCXEOGPPOQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.506781639 (84h33m4.377864694s) [noise=true , address=185.167.204.205:4005 , piece_id=VY7HILMJLYIMOF7XQ5JI2BYG3HL6CSC344FEX6526LEPLRAYNZXA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.507557626 (84h33m4.378640671s) [noise=true , address=185.167.204.206:4065 , piece_id=EK2QQCWVYSEYEB6TQUMRWIJL5QNKOCBNANOW5BMHP5TZMMILTK2Q, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.510750356 (84h33m4.381833401s) [noise=true , address=185.167.204.205:4028 , piece_id=O5YHC7OYSPVPFE6L4ME3QYLON4WOAH5AYHV2KRW3FYEYQLJVIYSQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:04.524221958 (84h33m4.395305003s) [noise=true , address=185.167.204.205:4016 , piece_id=CTPG2IQXMQWV2E7ECPR7EYPDZJPT7FX7JSEKAMGKBKJAJZEULRPQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.064989944 (84h33m4.936072989s) [noise=true , address=81.171.14.70:4043 , piece_id=3P3XPFAETWIVCJRB2UU5SZWURCVOGVEC3VXFA6FX4QMWQHLILTUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.087256625 (84h33m4.95833967s) [noise=true , address=81.171.14.70:4036 , piece_id=A6I4ODOTQY3PAWQSW2EKJYCCATULOSJR3CVTPQWSIEYGXKKUQHUA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.103625815 (84h33m4.97470886s) [noise=true , address=81.171.14.70:4027 , piece_id=2MVY2IXUDMNSLKZ4A2WEYKAOAOPY5IV354FFFICUCIHVKAOFL5HQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.110184184 (84h33m4.981267229s) [noise=true , address=81.171.14.70:4031 , piece_id=CT6IVGHABHFYX6FZZQBRNVU6YCKLF4Q7ZQIKGXKFBR4MMXMNLFWQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.536877270 (84h33m5.407960315s) [noise=true , address=81.171.14.70:4028 , piece_id=QUX4NVM26OQQAATGOOQ2MYNNK5N4T75UJGEFGEIYJ5DJ7QTQ2ZBA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.545643723 (84h33m5.416726768s) [noise=true , address=81.171.14.70:4048 , piece_id=PLTYOGFFQAXYY725S6HXLR3COY26JZYHMAJMCKJSOK5FGFDUW5DA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.547511790 (84h33m5.418594835s) [noise=true , address=81.171.14.70:4026 , piece_id=TWOWUTDDUN243MDF6VU5DPR3T233XGXD6XYUHNVV7LQT7IJ2QWPA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.549473886 (84h33m5.420556931s) [noise=true , address=81.171.14.70:4045 , piece_id=6GCAH2P7YMZR6E6KFQ4I3BKUCKW4XNNNRGZMKRQ46DRGHCB57R4A, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.649216977 (84h33m5.520300022s) [noise=true , address=81.171.14.70:4044 , piece_id=TL5E3CZQX37KXNA7RDK76B3ORGNRIGDUXX55XQAII4Y3PSJBSBFA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.736540175 (84h33m5.607623221s) [noise=true , address=89.149.201.147:4019 , piece_id=GM3THIX26QZBTAVKNSSNGF4MWUKH2NHN6QKQTAZR7I5AS7V55ZDQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739705292 (84h33m5.610788338s) [noise=true , address=89.149.201.147:4011 , piece_id=AG5C3J4XIHF5OCB74CWTDGAMHIST45QEYLXODF3FHPNOVWTAC4MQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: <nil>
Dec 9 04:02:05.739730531 (84h33m5.610813576s) [seg_pos={0 0}, upload=73832]: Segment reached optimal threshold of 26 pieces.
Dec 9 04:02:05.739992771 (84h33m5.611075836s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740003922 (84h33m5.611086967s) [noise=true , address=89.149.201.147:4003 , piece_id=DYWRYSYAE43FV5US46TQGX5P4WHZQNP766CMMRU4RXY6NBVZBHYA, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740059076 (84h33m5.611142121s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua): protocol: expected piece hash; context canceled; EOF
Dec 9 04:02:05.740088111 (84h33m5.611171157s) [noise=true , address=89.149.201.147:4018 , piece_id=ON3OVC4JXVG66QAJLDAEQXCELPECGWGF5IEZGDLUL2E23PH6DORQ, seg_pos={0 0}, upload=73832]: Long tail cancellation due to optimal threshold
Dec 9 04:02:05.740123359 (84h33m5.611206404s) [seg_pos={0 0}, upload=73832]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 04:02:05.740217909 (84h33m5.611300944s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 04:02:05.740225062 (84h33m5.611308107s) [upload=73832]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 04:05:41.230845175 (84h36m41.10192822s) [upload=73832]: Done uploading segments...
Dec 9 04:05:41.230955645 (84h36m41.102038691s) [upload=73832]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3m40.781237428s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.....................X......X.........................X.....] duration:
5.136901118s trace:
https://jaeger.dev.storj.io/trace/327471f8d037add8
FAIL (2025-12-09 04:05:34 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.13698117s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [....X............X.......................X...............X..] duration:
3.360696784s trace:
https://jaeger.dev.storj.io/trace/8c711ae8f1c72a8
FAIL (2025-12-09 04:05:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.360776276s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [.........X..XX....X.........X...................XX..........] duration:
5.70318111s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:05:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.703208073s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.....X............X.......................X...............X.] duration:
3.593301497s trace:
https://jaeger.dev.storj.io/trace/8c711ae8f1c72a8
FAIL (2025-12-09 04:05:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.593333239s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [......X............X.......................X...............X] duration:
5.000882765s trace:
https://jaeger.dev.storj.io/trace/8c711ae8f1c72a8
FAIL (2025-12-09 04:04:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000908544s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [......................X......X.........................X....] duration:
4.995730924s trace:
https://jaeger.dev.storj.io/trace/327471f8d037add8
FAIL (2025-12-09 04:04:39 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.995759188s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.......................X......X.........................X...] duration:
4.760767053s trace:
https://jaeger.dev.storj.io/trace/327471f8d037add8
FAIL (2025-12-09 04:04:09 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.760794324s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [..........X..XX....X.........X...................XX.........] duration:
5.910657765s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:04:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.910686931s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.......X............X.......................X...............] duration:
3.519520162s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 04:03:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.519537886s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [...........X..XX....X.........X...................XX........] duration:
5.939601985s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:03:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.939661809s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [............X..XX....X.........X...................XX.......] duration:
5.909374605s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:03:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.909399603s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [........X............X.......................X..............] duration:
3.541259069s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 04:03:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.54128505s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [.............X..XX....X.........X...................XX......] duration:
5.918372987s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:03:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.91840103s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [........................X......X.........................X..] duration:
4.893325812s trace:
https://jaeger.dev.storj.io/trace/327471f8d037add8
FAIL (2025-12-09 04:03:00 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.893346241s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [..............X..XX....X.........X...................XX.....] duration:
5.863501665s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:02:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.863527323s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.........X............X.......................X.............] duration:
3.52426019s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 04:02:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.524348067s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [..........X............X.......................X............] duration:
4.407743914s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 04:02:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.407772098s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [X..............X..XX....X.........X...................XX....] duration:
6.09606423s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:02:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.09609087s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........................X......X.........................X.] duration:
4.950638471s trace:
https://jaeger.dev.storj.io/trace/327471f8d037add8
FAIL (2025-12-09 04:01:57 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.950655475s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..........................X......X.........................X] duration:
5m18.587522187s trace:
https://jaeger.dev.storj.io/trace/327471f8d037add8
FAIL (2025-12-09 04:01:52 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5m18.587545353s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [.X..............X..XX....X.........X...................XX...] duration:
5.976032814s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:01:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.976059855s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [...........X............X.......................X...........] duration:
3.495999317s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 04:01:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.496025898s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [..X..............X..XX....X.........X...................XX..] duration:
5.768295375s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:01:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.768321574s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [...X..............X..XX....X.........X...................XX.] duration:
5.971654971s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:00:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.971668295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [X...X..............X..XX....X.........X...................XX] duration:
1m0.001340182s trace:
https://jaeger.dev.storj.io/trace/f8809ea99367e6f
FAIL (2025-12-09 04:00:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001352425s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [............X............X.......................X..........] duration:
3.35770074s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 04:00:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.357736387s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [.X...X..............X..XX....X.........X...................X] duration:
1m0.001308949s trace:
https://jaeger.dev.storj.io/trace/29df1d0c27569ef8
FAIL (2025-12-09 03:59:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001334559s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [X............X............X.......................X.........] duration:
3.380706213s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 03:59:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.380737491s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.X............X............X.......................X........] duration:
3.320268529s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 03:58:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.320298138s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..X............X............X.......................X.......] duration:
3.545251883s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 03:58:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.545280789s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [..X...X..............X..XX....X.........X...................] duration:
5.778395629s trace:
https://jaeger.dev.storj.io/trace/684a903240bd32e0
FAIL (2025-12-09 03:57:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.778423602s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...X............X............X.......................X......] duration:
3.503401161s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 03:57:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.503428053s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [...X...X..............X..XX....X.........X..................] duration:
5.844050157s trace:
https://jaeger.dev.storj.io/trace/684a903240bd32e0
FAIL (2025-12-09 03:56:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.844075003s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [....X............X............X.......................X.....] duration:
3.447163714s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 03:56:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.447189622s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................X......X.........................] duration:
5.041651895s trace:
https://jaeger.dev.storj.io/trace/73717879a82ed6ab
FAIL (2025-12-09 03:55:38 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.041680079s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [....X...X..............X..XX....X.........X.................] duration:
5.939338422s trace:
https://jaeger.dev.storj.io/trace/684a903240bd32e0
FAIL (2025-12-09 03:55:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.939367379s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.....X............X............X.......................X....] duration:
3.533925508s trace:
https://jaeger.dev.storj.io/trace/4adf4aa74ca8b5ee
FAIL (2025-12-09 03:55:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.533991484s
core
uplink
us-select-1
us1-placement-20
|