Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.016896234s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:48:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.016914939s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 5.271083663s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:47:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.271110984s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 5.456817515s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:47:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.456842864s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.038496233s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:47:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.038529708s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.068067105s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:46:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.068093116s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 6.10030385s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:46:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.100330821s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.85189563s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:46:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.852455897s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 5.695130009s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:45:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.695154767s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 4.661931178s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:45:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 4.661962598s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.912947025s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:44:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.912972883s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.959775846s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:44:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.959804882s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 5.065587659s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:44:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.065607317s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.929893376s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:44:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.929923322s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 5.663017369s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:43:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.663045312s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.163969007s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:43:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.16401744s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 5.847661681s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:43:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.847688952s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.990588768s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:43:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.990634807s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 5.677748893s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:42:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.677777398s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.973192315s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:42:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.973229414s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 5.54430885s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:42:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.544338397s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.884432179s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:42:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.884462476s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 5.651041267s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:41:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.651074011s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.99381998s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:41:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.993846079s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.973545266s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:40:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.973606963s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 5.779204661s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:40:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.779225922s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 5.162989761s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:40:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.163015689s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.957731033s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:40:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.957765579s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 5.661637644s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:39:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.661665757s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.135796807s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:39:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.135828458s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 5.62680829s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:38:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.626861943s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 5.71033421s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:38:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.710361171s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.006437389s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:38:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.00647022s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 5.740050224s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:37:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.740077254s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.930799282s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:37:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.930818357s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 5.329752412s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:36:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.329782338s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.03776651s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:36:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.037843858s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 5.690420912s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:35:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.690448364s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.967092093s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:35:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.967138212s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 5.754170991s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:34:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.754200697s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 5.670044702s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:34:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.670077605s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.000568108s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:34:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.000601431s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.142105921s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:33:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.142155595s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 5.67528323s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:33:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.675315601s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 5.682002235s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:32:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.682030929s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.942397498s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:32:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.942425501s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 6.292426887s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:31:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.292454949s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.906932646s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:31:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.90696104s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.971739434s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:30:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.971781925s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.130366814s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:30:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.130409516s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 5.673609122s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:30:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.673631454s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)