Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.261038848s FAIL (2025-06-27 20:03:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.261061911s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.253821932s FAIL (2025-06-27 20:03:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.253862551s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.354950646s FAIL (2025-06-27 20:02:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.354975453s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.255371816s FAIL (2025-06-27 20:01:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.25539499s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.360121804s FAIL (2025-06-27 20:00:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.360143596s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.369949698s FAIL (2025-06-27 20:00:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.369963374s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.287262768s FAIL (2025-06-27 20:00:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.287281024s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.37532951s FAIL (2025-06-27 19:59:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.375353856s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.232304636s FAIL (2025-06-27 19:58:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.232327089s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.497847326s FAIL (2025-06-27 19:57:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.497869558s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.244511882s FAIL (2025-06-27 19:56:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.244533424s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.294430198s FAIL (2025-06-27 19:55:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.294464355s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.360707053s FAIL (2025-06-27 19:55:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.360729566s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.231529001s FAIL (2025-06-27 19:54:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.231550383s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.354212382s FAIL (2025-06-27 19:53:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.354234965s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.300184718s FAIL (2025-06-27 19:52:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.300206951s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.305277507s FAIL (2025-06-27 19:51:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.305298527s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.552332866s FAIL (2025-06-27 19:49:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.552362292s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.18305803s FAIL (2025-06-27 19:48:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.183082156s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.224066315s FAIL (2025-06-27 19:47:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.224077928s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000876427s FAIL (2025-06-27 19:47:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000902657s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.745469464s FAIL (2025-06-27 19:44:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.745491535s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.104703236s FAIL (2025-06-27 19:44:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.104734635s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.241536006s FAIL (2025-06-27 19:43:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.241567926s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.546103817s FAIL (2025-06-27 19:42:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.546129966s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.710289363s FAIL (2025-06-27 19:41:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.710327575s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.731310024s FAIL (2025-06-27 19:40:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.731333519s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.365553134s FAIL (2025-06-27 19:40:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.365587902s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.66973318s FAIL (2025-06-27 19:39:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.6697592s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.68462254s FAIL (2025-06-27 19:38:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.684644261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.415294629s FAIL (2025-06-27 19:37:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.415320289s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.805505037s FAIL (2025-06-27 19:37:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.805529775s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.127337493s FAIL (2025-06-27 19:36:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.127364964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.801349273s FAIL (2025-06-27 19:34:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.801381715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.546910989s FAIL (2025-06-27 19:33:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.546941907s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.186607189s FAIL (2025-06-27 19:32:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.186637486s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.985185112s FAIL (2025-06-27 19:32:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.985215029s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.183673997s FAIL (2025-06-27 19:31:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.18370246s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.694991413s FAIL (2025-06-27 19:31:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.695020008s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.782810687s FAIL (2025-06-27 19:30:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.782825977s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.895813156s FAIL (2025-06-27 19:29:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.895839497s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.080719947s FAIL (2025-06-27 19:28:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.080741708s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.522624904s FAIL (2025-06-27 19:27:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.522652578s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.14152433s FAIL (2025-06-27 19:27:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.141549208s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.01253666s FAIL (2025-06-27 19:26:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.012564492s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.806124486s FAIL (2025-06-27 19:25:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.806168471s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.335325336s FAIL (2025-06-27 19:24:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.33535431s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.243113473s FAIL (2025-06-27 19:24:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.24316431s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.541964997s FAIL (2025-06-27 19:23:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.541994164s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.063338169s FAIL (2025-06-27 19:22:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.063365732s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)