Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.108223456s FAIL (2025-10-02 07:09:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.108254224s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.815059844s FAIL (2025-10-02 07:09:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.815090482s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.099050808s FAIL (2025-10-02 07:09:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.099082548s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.699723327s FAIL (2025-10-02 07:09:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.699778954s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.654082355s FAIL (2025-10-02 07:09:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.654116569s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.435958755s FAIL (2025-10-02 07:08:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.435976338s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 7.01712356s FAIL (2025-10-02 07:08:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 7.017184205s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.647943478s FAIL (2025-10-02 07:08:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.647966001s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.187123074s FAIL (2025-10-02 07:07:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.187172678s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.062826905s FAIL (2025-10-02 07:07:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.062854035s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.863134376s FAIL (2025-10-02 07:07:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.863222493s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.447591216s FAIL (2025-10-02 07:07:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.447618057s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.159830774s FAIL (2025-10-02 07:07:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.159870411s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.653277644s FAIL (2025-10-02 07:07:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.653410377s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.537163617s FAIL (2025-10-02 07:06:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.537202392s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.206680435s FAIL (2025-10-02 07:06:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.206718517s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.744392013s FAIL (2025-10-02 07:05:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.74448002s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.117816875s FAIL (2025-10-02 07:05:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.117839649s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.228382018s FAIL (2025-10-02 07:05:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.228408229s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.003195273s FAIL (2025-10-02 07:04:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.003226804s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.70771452s FAIL (2025-10-02 07:04:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.707743135s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.35776704s FAIL (2025-10-02 07:04:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.357803138s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.15667401s FAIL (2025-10-02 07:04:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.15671016s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.081929396s FAIL (2025-10-02 07:04:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.081963231s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.201643012s FAIL (2025-10-02 07:03:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.201681356s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.706187164s FAIL (2025-10-02 07:03:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.706219344s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.965568249s FAIL (2025-10-02 07:03:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.965607493s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.784416914s FAIL (2025-10-02 07:02:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.784444227s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.103445569s FAIL (2025-10-02 07:02:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.103476919s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.397924746s FAIL (2025-10-02 07:02:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.397950985s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.088447909s FAIL (2025-10-02 07:01:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.088543921s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.507992851s FAIL (2025-10-02 07:01:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.508017277s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.690230538s FAIL (2025-10-02 07:01:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.690257259s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.295960398s FAIL (2025-10-02 07:01:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.295994863s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.464048745s FAIL (2025-10-02 07:00:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.464075707s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.690543996s FAIL (2025-10-02 07:00:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.690595394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.694904463s FAIL (2025-10-02 07:00:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.694929381s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.449577409s FAIL (2025-10-02 07:00:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.449608368s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.494890902s FAIL (2025-10-02 06:59:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.494920069s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.991752626s FAIL (2025-10-02 06:59:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.991782182s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.738354767s FAIL (2025-10-02 06:59:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.738387369s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.307134138s FAIL (2025-10-02 06:59:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.307161319s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.713364936s FAIL (2025-10-02 06:58:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.713389343s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.180155468s FAIL (2025-10-02 06:58:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.180302969s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.14543356s FAIL (2025-10-02 06:58:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.145458598s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.258981034s FAIL (2025-10-02 06:58:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.259017123s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.902044048s FAIL (2025-10-02 06:57:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.902133137s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.020277803s FAIL (2025-10-02 06:57:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 2 06:45:01.165728558 (209h13m46.627821051s) [upload=183988]: Got next segment. Inline: false Oct 2 06:45:01.165773403 (209h13m46.627865886s) [upload=183988]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 2 06:45:01.165780918 (209h13m46.627873401s) [upload=183988]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 2 06:45:01.431617535 (209h13m46.893710028s) [seg_pos={0 0}, upload=183988]: Begin upload segment... Oct 2 06:45:01.431697767 (209h13m46.89379025s) [seg_pos={0 0}, upload=183988]: Done begin upload segment. Oct 2 06:45:01.431733565 (209h13m46.893826048s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.431781507 (209h13m46.89387399s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.432603612 (209h13m46.894696095s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433107522 (209h13m46.895199995s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.433549994 (209h13m46.895642477s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.434770508 (209h13m46.896862991s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435293313 (209h13m46.897385796s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.435758981 (209h13m46.897851464s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.436566698 (209h13m46.898659182s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437037004 (209h13m46.899129487s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437501258 (209h13m46.899593741s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.437998455 (209h13m46.900090938s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439162861 (209h13m46.901255345s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.439750080 (209h13m46.901842563s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.441380735 (209h13m46.903473208s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:01.443563041 (209h13m46.905655524s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Uploading piece... Oct 2 06:45:03.164025603 (209h13m48.626118096s) [upload=183988]: Next returned nil segment Oct 2 06:45:03.164074216 (209h13m48.626166699s) [upload=183988]: Waiting for error group managing segments... Oct 2 06:45:03.564517516 (209h13m49.026609999s) [noise=true , address=45.91.116.6:4975 , piece_id=ZNB45P2Q3Q3LUS3DG4FPV5IHHBGSCE5VBCXCW3HFL56GN6EKO7JQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.610247563 (209h13m49.072340046s) [noise=true , address=77.90.31.21:4878 , piece_id=6S3KSKZN7QKWQCSAOJHVNCLET7JBWPZIRNFQI2SX3UKDMT732KLA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.616241056 (209h13m49.078333549s) [noise=true , address=45.91.116.11:4118 , piece_id=KBHSEEDCAKKH76W7FBETEES4UXAMTTA2AQAHETDNJ6ZND73X2OGQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.632371971 (209h13m49.094464475s) [noise=true , address=77.90.31.12:4589 , piece_id=OVKIKUBOCPLT3VJVV4ML6QIQLGOA2S4AIBUUR5IYKAUE2IQVUDGA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.639558567 (209h13m49.10165105s) [noise=true , address=45.91.116.12:4227 , piece_id=MGYI2TR4MPA6DKGFYIVMGTGLD5DO5BROGFCKWOP4Z4RNRWOSBPMQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.649768408 (209h13m49.111860891s) [noise=true , address=77.90.31.11:4554 , piece_id=J5OKH5FR5YIBUZEN55CPSKDJ7L3O7PGF5VDS2X3C7CGTBCXUEVUA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.654368487 (209h13m49.11646099s) [noise=true , address=45.91.116.5:4299 , piece_id=2YUDNHDTW3DDWZJPHUV4AHHAO6GJI3TVUKMXO3JDBDIG3YB5XLEA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.688906413 (209h13m49.150998906s) [noise=true , address=45.91.116.4:4939 , piece_id=MGTGITWIRGG53CWAY7276LOHSVJGACIAAMSISMKPZGJLECG52PLQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.706565178 (209h13m49.168657671s) [noise=true , address=45.91.116.8:4047 , piece_id=ZSWJVTA7HW7U2IHRKGKDUHNYCCSMK3YJ3KJBQLOIQ773XCGLWV3Q, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.744669075 (209h13m49.206761558s) [noise=true , address=45.91.116.13:4155 , piece_id=JMRFSKD77UQKZXUZIFCQPAAVCNFDNGNV7U2SKSAJDOKARS6NKA5A, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:03.767094667 (209h13m49.22918715s) [noise=true , address=45.91.116.8:4048 , piece_id=FVQDCO7EYOYOAOORYY7DAQYIAXWBBDN7OF7E5B2Z3MT34QUGGLUQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: <nil> Oct 2 06:45:30.458020428 (209h14m15.920112912s) [noise=true , address=145.79.227.13:4801 , piece_id=ZL7KE3QVM63VCBNMAUEXELEERW2JC65RTCVUUQKYLFTEJZBZPRVA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42996->145.79.227.13:4801: use of closed network connection Oct 2 06:45:30.458236099 (209h14m15.920328572s) [noise=true , address=145.79.227.5:4221 , piece_id=SWGYPEIGFBLMMVAUFNIEQ63SL5SEDXFOCGIFCB7C7ZNCDYDZS7BQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42060->145.79.227.5:4221: use of closed network connection Oct 2 06:45:30.458311102 (209h14m15.920403585s) [noise=true , address=145.79.227.2:4115 , piece_id=DZADXAOVVZSNVIMHVE5N7XA4U7Y2RS6U6JQLTC2X6YSWWSGVP6YQ, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55776->145.79.227.2:4115: use of closed network connection Oct 2 06:45:30.458404109 (209h14m15.920496592s) [noise=true , address=145.79.227.11:4730 , piece_id=QEESDYHFF4TTJ6DOW5BAXORGOSWAKLVFDUWMYZT5POSJ6J5D7IOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47498->145.79.227.11:4730: use of closed network connection Oct 2 06:45:30.458519669 (209h14m15.920612142s) [noise=true , address=145.79.227.16:4909 , piece_id=DVOXCRCAGC43BIDHGJOGQBZLDM4QBUONXGWF3GVVUZ4V5YSC6NOA, seg_pos={0 0}, upload=183988]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36010->145.79.227.16:4909: use of closed network connection Oct 2 06:45:30.458566669 (209h14m15.920659152s) [seg_pos={0 0}, upload=183988]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 2 06:45:30.710989853 (209h14m16.173082346s) [upload=183988]: Done uploading segments... Oct 2 06:45:30.711022285 (209h14m16.173114778s) [upload=183988]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.020309685s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.038337775s FAIL (2025-10-02 06:57:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.038374043s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.48672425s FAIL (2025-10-02 06:57:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.486758896s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)