Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.945034064s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:09:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.945068299s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.100941556s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:09:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.100978136s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.099982722s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:08:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.100010565s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.008309549s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:07:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.008392887s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.99622358s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:06:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.996253647s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.026785708s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:05:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.026811648s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.938258932s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:04:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.938278298s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.138466402s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:04:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.138504153s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.89487911s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:04:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.894912913s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.121811876s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:03:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.121835462s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.087340097s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 02:02:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.087371257s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.093319707s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:02:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.093369803s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.967853931s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:02:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.967910288s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.522519627s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 02:02:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.522550366s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 5.787518562s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 02:01:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.787551485s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.010964741s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:01:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.010992553s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.989064349s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:01:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.989097022s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.704916433s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 02:00:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.704943364s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.179883438s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:00:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.179913957s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 5.482895614s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 02:00:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.482929879s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 7.369669298s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 02:00:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.369732839s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 5.754528924s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:59:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.754558511s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 5.291749233s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:59:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.291775023s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 5.711605829s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:58:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.71163236s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.930484301s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:58:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.930517203s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 5.865416442s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:58:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.865443202s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 6.147839293s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:58:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.147874128s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.021767955s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:57:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.021798092s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 5.625316588s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:57:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.625344492s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.063591774s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:56:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.063619547s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 5.953550172s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:56:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.953583995s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.963719283s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:56:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.96374829s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 5.709872409s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:55:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.70990415s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.9014987s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:55:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.901523788s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.995200924s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:54:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.99522932s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 5.715758268s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:54:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.715786953s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.965574729s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:54:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.965602s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 8.465368922s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:54:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 8.465402976s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 5.666957336s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:53:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.666984948s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.005198127s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:53:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.005225419s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 5.719663956s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:52:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.719694104s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.108406955s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:52:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.108433666s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 5.921502176s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:52:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.921533516s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 5.689190299s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:52:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.689219094s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.984661123s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:51:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.984686881s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 5.819499581s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:51:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.819529799s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 5.798964944s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:51:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.799032333s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.032386812s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:50:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.032413835s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.018652747s trace: https://jaeger.dev.storj.io/trace/5242724484c3891b FAIL (2025-11-16 01:50:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.018680531s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 5.772510377s trace: https://jaeger.dev.storj.io/trace/2e91e72686ae0b40 FAIL (2025-11-16 01:49:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== 1736020 (161h19m8.793206471s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242215683 (161h19m8.793686134s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.242742376 (161h19m8.794212837s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243252758 (161h19m8.794723219s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.243840567 (161h19m8.795311019s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.244451301 (161h19m8.795921762s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245056383 (161h19m8.796526844s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.245583326 (161h19m8.797053787s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:10.246207625 (161h19m8.797678086s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Uploading piece... Nov 16 01:23:11.449262278 (161h19m10.000732739s) [upload=142017]: Next returned nil segment Nov 16 01:23:11.449274801 (161h19m10.000745252s) [upload=142017]: Waiting for error group managing segments... Nov 16 01:23:12.019429678 (161h19m10.570900139s) [noise=true , address=185.167.204.205:4034 , piece_id=G7AWLCKDBWLXH5LXHWR2YXA7TFPKFASHX64F6ON6JIROUEN4MGVA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.020767936 (161h19m10.572238398s) [noise=true , address=185.167.204.205:4022 , piece_id=YNI5CY6ONT4DRMO2FAZOYENUQK5HO4VBUKGAKSYQHUEBSSB3LCNQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.024358022 (161h19m10.575828484s) [noise=true , address=185.167.204.205:4005 , piece_id=DQSBZDXK5ZPOI4PKHVBG6YDM437B53CDF6AXCDVRMJ3IMIM6CBOQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.029914362 (161h19m10.581384823s) [noise=true , address=185.167.204.206:4071 , piece_id=43642EVRD4SVMU4GYIUN2FSDSALPYG4SXJJTU5DTGJGI7JBMVJ7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.030581994 (161h19m10.582052445s) [noise=true , address=185.167.204.206:4053 , piece_id=P55GMLUCGOTNM7H5YIEXAAFK2ANL3GKNFJFX2BYCEA7UXGFPDRDQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.035155813 (161h19m10.586626264s) [noise=true , address=185.167.204.206:4070 , piece_id=OUWMYK3LF2VBLWX52RQEB7NSL2DAZXXCYVV6ANLLSCP43ZZMBCJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.050092807 (161h19m10.601563268s) [noise=true , address=185.167.204.206:4037 , piece_id=7C7ATQ27TKZYDHIFPXX2K76ORR2PASTIFGBTDROLMEO4IWH6HMJA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.053686641 (161h19m10.605157102s) [noise=true , address=185.167.204.206:4039 , piece_id=PKH4E6SLSEQWGSCRB5HUMS4C3H7G7UPCPUXES2F72ZLQ5XH5JMFQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.060220813 (161h19m10.611691274s) [noise=true , address=185.167.204.206:4056 , piece_id=KGWQ65236SGFBKSZ6P5OLKEKGTGOABZCFKZKPCTK2X7ZKWYZRSJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.064116672 (161h19m10.615587133s) [noise=true , address=185.167.204.205:4015 , piece_id=KPNVK3CRPHA23SKOQB47FQKP2IZ6SIFPHNV7X7USX6BNVGRF62ZA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.070522139 (161h19m10.6219926s) [noise=true , address=185.167.204.205:4013 , piece_id=4GQINGWNOGIMNOC4NRD57ECY32ZW5YGQQXL47KAVLM7A5VCPJ3AQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.600587265 (161h19m11.152057736s) [noise=true , address=89.149.201.147:4011 , piece_id=HOV3AHSGQGCMXQAOMVHO4VCXI4SBFLCX7AWLYISE2WS23G5YKVZQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.624319988 (161h19m11.175790449s) [noise=true , address=89.149.201.147:4003 , piece_id=CVJVSGREBXPIM3IEO6JMITNTESM74OWTLJCT2AZGPFKIBGVVAT4A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.645331518 (161h19m11.196801979s) [noise=true , address=89.149.201.147:4016 , piece_id=L222WH4GWW6G6FTAGNJG4TVPD3ALHONWKL5OGR42JZNAAYJUSZGA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.647723873 (161h19m11.199194334s) [noise=true , address=89.149.201.147:4009 , piece_id=FNGJETIXT3VKWU7UVXLGUSKBXN5TQLKIEMA4JIGGAYHCLDRQWW7Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.662279581 (161h19m11.213750042s) [noise=true , address=89.149.201.147:4010 , piece_id=BWN2GTSNBKF5EOW6BST3XSAQPNEG6MXCIKLTANFQZ4CMF6YNGWKA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.676860756 (161h19m11.228331217s) [noise=true , address=89.149.201.147:4019 , piece_id=LRHLWIIIFSJQFPIVBNCF37PF5Q4AGBMTSKXVYW2FXI4YAPMHF4FQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.699008159 (161h19m11.25047862s) [noise=true , address=89.149.201.147:4002 , piece_id=MDI45R7RCCO6DA22666UKZWM7WJSH5EKS6TOI5I2D4SICNXFRIYA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.706951986 (161h19m11.258422448s) [noise=true , address=89.149.201.147:4024 , piece_id=DZMF2F24IOSVO5JUHU3JYFCN2Z64Y7FRJFK7MULYPIWYS43MC6NQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:23:12.765782933 (161h19m11.317253384s) [noise=true , address=89.149.201.147:4020 , piece_id=4AJAOWXCF2ESA3AWSNOAI57B6GZE42KGCE6UGVC2AXI2B26SEC4Q, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:06.407083616 (161h20m4.958554087s) [noise=true , address=81.171.14.70:4027 , piece_id=6WLY3OLIYVEYSW3YN4ZRJMHPZ5IXB7W2GV7YRTMVWN4YBBCOYHXA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:08.566900019 (161h20m7.11837048s) [noise=true , address=81.171.14.70:4034 , piece_id=5VKATBUZ7Y2ISCSRX367MTEYDWZYYCHODZAQRQW4HW63VICHZQJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.011480464 (161h20m7.562950926s) [noise=true , address=81.171.14.70:4042 , piece_id=J5PBO2JY2YIAQ4UZKYX6APPE5HXUKKIG5C7L5QKEGUQ7QG4LR2MA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: <nil> Nov 16 01:24:09.579285615 (161h20m8.130756076s) [noise=true , address=81.171.14.70:4036 , piece_id=3NKWIA5KFZQL6I6IY7WXGXB424IVNCWYOWETSB6PK4HXSNSUGFXQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54366->81.171.14.70:4036: use of closed network connection Nov 16 01:24:09.579597900 (161h20m8.131068361s) [noise=true , address=81.171.14.70:4045 , piece_id=6XTPL2OT76CWEFECBTSQUWWTXZLVYNRYTVCS3FFF2R7OPWPQLT3A, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.579822729 (161h20m8.13129319s) [noise=true , address=81.171.14.70:4040 , piece_id=KPFPVAGMVKYUMS5XFP7FDLSQTAZ73UKXHWCDPXVNBW6FELJKI7HA, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: expected piece hash; context deadline exceeded; EOF Nov 16 01:24:09.580145032 (161h20m8.131615483s) [noise=true , address=81.171.14.70:4026 , piece_id=NGFRG2NHKW6HKGODXYILSENB3QFLGQWFVKV2K7K2FSPVXN6L7AJQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 188.166.225.170:34976->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 16 01:24:09.580189147 (161h20m8.131659618s) [noise=true , address=81.171.14.70:4029 , piece_id=FQ6XP24EO63LX4CVNUCVEJVJXKV63BQ3V2Q2KQQYI3KG7O5JKQUQ, seg_pos={0 0}, upload=142017]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52482->81.171.14.70:4029: use of closed network connection Nov 16 01:24:09.580283806 (161h20m8.131754257s) [seg_pos={0 0}, upload=142017]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Nov 16 01:24:09.751173326 (161h20m8.302643787s) [upload=142017]: Done uploading segments... Nov 16 01:24:09.751217872 (161h20m8.302688323s) [upload=142017]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.772544452s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)