Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 6.110524726s FAIL (2025-10-09 07:37:50 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.110555736s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 10.730875047s FAIL (2025-10-09 07:37:02 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 10.730912288s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 6.054767756s FAIL (2025-10-09 07:35:56 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.0548957s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 6.217067889s FAIL (2025-10-09 07:35:18 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.217104007s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 6.525971093s FAIL (2025-10-09 07:34:35 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.526005969s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 7.506250759s FAIL (2025-10-09 07:33:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 7.506286128s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 6.050266944s FAIL (2025-10-09 07:33:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.050303004s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 8.926890347s FAIL (2025-10-09 07:32:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 8.926921937s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 6.279268623s FAIL (2025-10-09 07:31:41 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.279293711s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 6.92322429s FAIL (2025-10-09 07:30:57 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.923256741s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 7.282015758s FAIL (2025-10-09 07:30:27 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 7.282052148s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 6.277928364s FAIL (2025-10-09 07:29:59 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.277958762s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 9.868543449s FAIL (2025-10-09 07:29:33 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 9.868577996s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 6.297844256s FAIL (2025-10-09 07:28:25 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.297879513s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 6.489169943s FAIL (2025-10-09 07:28:07 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.489206453s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 13.960749189s FAIL (2025-10-09 07:27:03 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 13.960788354s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 7.205784638s FAIL (2025-10-09 07:26:40 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 7.205818884s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 7.202455912s FAIL (2025-10-09 07:25:49 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 7.202497971s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 11.923287387s FAIL (2025-10-09 07:25:20 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 11.923325439s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 6.230892392s FAIL (2025-10-09 07:24:47 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.230919773s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 10.411378463s FAIL (2025-10-09 07:24:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 10.411417687s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 7.929898118s FAIL (2025-10-09 07:23:33 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 7.929932643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 7.053633485s FAIL (2025-10-09 07:22:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 7.053671607s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 9.609077128s FAIL (2025-10-09 07:21:41 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 9.609108127s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 6.434297223s FAIL (2025-10-09 07:20:24 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.434332791s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 6.440473473s FAIL (2025-10-09 07:19:22 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.440538496s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 16.299402446s FAIL (2025-10-09 07:18:41 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 16.299432232s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 13.078349751s FAIL (2025-10-09 07:18:18 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 13.078393054s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 6.927060738s FAIL (2025-10-09 07:17:30 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.927095124s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 6.457107899s FAIL (2025-10-09 07:16:45 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.45714012s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 6.408421592s FAIL (2025-10-09 07:16:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 6.408453012s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 12.895356757s FAIL (2025-10-09 07:15:13 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 12.895391132s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 8.885127274s FAIL (2025-10-09 07:14:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 8.88516214s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 16.901286235s FAIL (2025-10-09 07:13:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 16.901316964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.003001334s FAIL (2025-10-09 07:13:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 9 07:12:45.828340145 (377h41m31.290432618s) [upload=332294]: Starting upload Oct 9 07:12:45.828479781 (377h41m31.290572264s) [upload=332294]: Done starting upload Oct 9 07:12:45.828554443 (377h41m31.290646956s) [upload=332294]: Uploading segments... Oct 9 07:12:45.828768019 (377h41m31.290860503s) [upload=332294]: Got next segment. Inline: false Oct 9 07:12:45.828809308 (377h41m31.290901801s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:45.828816862 (377h41m31.290909335s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:46.090489688 (377h41m31.552582181s) [seg_pos={0 0}, upload=332294]: Begin upload segment... Oct 9 07:12:46.090589387 (377h41m31.55268188s) [seg_pos={0 0}, upload=332294]: Done begin upload segment. Oct 9 07:12:46.090624845 (377h41m31.552717328s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.090679299 (377h41m31.552771792s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091424579 (377h41m31.553517062s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.091884314 (377h41m31.553976797s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092309433 (377h41m31.554401916s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.092844663 (377h41m31.554937146s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093269552 (377h41m31.555362035s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.093686395 (377h41m31.555778869s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094126343 (377h41m31.556218826s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094550060 (377h41m31.556642543s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.094963848 (377h41m31.557056321s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.095757198 (377h41m31.557849691s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.099911409 (377h41m31.562003902s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.102140854 (377h41m31.564233337s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.103684224 (377h41m31.565776707s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:46.104411879 (377h41m31.566504362s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Uploading piece... Oct 9 07:12:47.857988130 (377h41m33.320080613s) [upload=332294]: Next returned nil segment Oct 9 07:12:47.858008148 (377h41m33.320100631s) [upload=332294]: Waiting for error group managing segments... Oct 9 07:12:48.148813408 (377h41m33.610905891s) [noise=true , address=145.79.227.5:4222 , piece_id=ZATNOVM64CALOBO5VJP7AAETBY4ZBJDI3M4ZV3LJIIYXWBIO5HPQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.151017424 (377h41m33.613109907s) [noise=true , address=145.79.227.6:4259 , piece_id=6FC5FLSW3PRZM72K4I2WS6OAEI62JZR67UYYLHY4IZBO5L5SQNIQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.153545198 (377h41m33.615637681s) [noise=true , address=145.79.227.11:4729 , piece_id=ITJGHTRMCAIEU4TCK2HPAJM4DPIFMENSJGQQAH4WYKAE7X75XPGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.155438955 (377h41m33.617531428s) [noise=true , address=145.79.227.15:4873 , piece_id=QXM6H4YCWTLYZKPT72ZLJKOOZYPWP5CUYGMC5JYSUNQOTUR3ME4A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.182802090 (377h41m33.644894563s) [noise=true , address=145.79.227.17:4970 , piece_id=6PDO3L72JRVE26N3RH674BVQBR3CQRFAWW5IVNMK4443DIAI33GQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.298451002 (377h41m33.760543475s) [noise=true , address=77.90.31.16:4733 , piece_id=I672MP7CCLMDQILTL577QC2R3KA5GAZFVQ62HIHDTYL7FF2ZZPQQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.305377401 (377h41m33.767469884s) [noise=true , address=45.91.116.12:4227 , piece_id=Z5BIEIKMCQ6AHQQMYD7N6A2Z6DNCFSBOGG555QUIQOBLTSV46EYQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.328760708 (377h41m33.790853201s) [noise=true , address=45.91.116.6:4973 , piece_id=HX6GPHVIK5YRK5AVAVI3CCY25QQ5TYYKKYVGOA7P7OV6IYCEAT2Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.330883801 (377h41m33.792976284s) [noise=true , address=45.91.116.7:4009 , piece_id=GMLOCUCHJ2CSJAVMIWELRAMN4LWCCGJFMERVA4LVL6V7RTIPTOIA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.331368294 (377h41m33.793460767s) [noise=true , address=45.91.116.13:4154 , piece_id=7DMFRPIHNNO3WKWLA5WRU7GJLZZNIU67SXSMCX3SZE42KHSPEA7A, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.335337461 (377h41m33.797429945s) [noise=true , address=45.91.116.14:4191 , piece_id=HIGE4LT4CJXWXT4AJV6SDV7GYO2UY74XFKRKEVVGQJ5UO436GZ4Q, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.337698137 (377h41m33.79979062s) [noise=true , address=45.91.116.8:4048 , piece_id=JUMW4QHDQVLSHHXPSJXBGKJYTM7QLPOEQOEUSMTAUEDB52JSBJHQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386474288 (377h41m33.848566771s) [noise=true , address=77.90.31.14:4662 , piece_id=6MHYLSYLTCA3CKRKOLZRNXMI4OK5U6EJ3FXO7WNBWJ4FGTLYKYWA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: <nil> Oct 9 07:12:48.386512040 (377h41m33.848604523s) [seg_pos={0 0}, upload=332294]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:48.386662387 (377h41m33.84875487s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386672856 (377h41m33.848765339s) [noise=true , address=77.90.31.10:4517 , piece_id=GZGCWGBHNVLPMIXMCZS2E232744BC2NYEQ3WBXOXUYTCMD33OHGQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386778898 (377h41m33.848871391s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386787124 (377h41m33.848879607s) [noise=true , address=45.91.116.9:4262 , piece_id=XXN6GHX7GJDZWCAEYZ2IHX4DEHRFRKYYMNSJKLQKVQJS3CNYNVSA, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386842008 (377h41m33.848934492s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:48.386866305 (377h41m33.848958778s) [noise=true , address=45.91.116.10:4083 , piece_id=IQ7FWBKVZ7J77K2DKD66KL5SEZR434TE47K2OART7MC7Y3M2AFWQ, seg_pos={0 0}, upload=332294]: Long tail cancellation due to optimal threshold Oct 9 07:12:48.386914737 (377h41m33.84900722s) [seg_pos={0 0}, upload=332294]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:48.387031108 (377h41m33.849123591s) [upload=332294]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:48.387037942 (377h41m33.849130415s) [upload=332294]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:48.860210039 (377h41m34.322302532s) [upload=332294]: Done uploading segments... Oct 9 07:12:48.860237832 (377h41m34.322330315s) [upload=332294]: Upload finished. err: <nil> Duration: 30.003029399s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002849456s FAIL (2025-10-09 07:12:39 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 9 07:12:10.438536887 (377h40m55.90062937s) [upload=332286]: Starting upload Oct 9 07:12:10.438656957 (377h40m55.90074945s) [upload=332286]: Done starting upload Oct 9 07:12:10.438784208 (377h40m55.900876681s) [upload=332286]: Uploading segments... Oct 9 07:12:10.438968079 (377h40m55.901060562s) [upload=332286]: Got next segment. Inline: false Oct 9 07:12:10.439004198 (377h40m55.901096671s) [upload=332286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 9 07:12:10.439009438 (377h40m55.901101921s) [upload=332286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 9 07:12:10.849709294 (377h40m56.311801777s) [seg_pos={0 0}, upload=332286]: Begin upload segment... Oct 9 07:12:10.849785729 (377h40m56.311878212s) [seg_pos={0 0}, upload=332286]: Done begin upload segment. Oct 9 07:12:10.849887513 (377h40m56.311979996s) [noise=true , address=145.79.227.12:4767 , piece_id=WMWS3EMQ2SDPXMKIL3MKYQ53F5Z2K4M62KXBAIX5I6WU45Z3PR4A, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.849909205 (377h40m56.312001688s) [noise=true , address=45.91.116.9:4263 , piece_id=7GXU2W3IIJRIDN5TXHRVIWALRLNGD5YLIH725AB6C2L34GN4ZCEQ, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.850637061 (377h40m56.312729544s) [noise=true , address=45.91.116.5:4299 , piece_id=P3RENZ45B2IIQTFJYUNM6BP4A4CPPLTX3QXISNH3KVXUZQBQDV7Q, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.850720980 (377h40m56.312813463s) [noise=true , address=145.79.227.5:4222 , piece_id=GTKYCUZMRS76ENMWX7HSPUHU7RJQWKP34RBW2N6WZCEJ7OYDDM7A, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.850983651 (377h40m56.313076134s) [noise=true , address=45.91.116.12:4225 , piece_id=CDX7UWHLKYYC5F2VEH5PMMCSBMRKPYXNQZUD3IOOUIFOV6T7RMCQ, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.851337925 (377h40m56.313430408s) [noise=true , address=145.79.227.19:4079 , piece_id=U2OBNAMDOXJZ6TLDOBZTLMBZ5CQZIIVNGALVQQAYYWUAT5NB4NPA, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.851386297 (377h40m56.31347878s) [noise=true , address=145.79.227.18:4006 , piece_id=QAZMOFEQHLLXP43OSHIN5ZPYIKPHUAGJBO44DFLJ6H7QUU7GIGYA, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.851770489 (377h40m56.313862972s) [noise=true , address=45.91.116.14:4189 , piece_id=KR46A5F2UWLK7VWDPVVL2RLYENHVY5GMA6HYBRQI7XY3MYLZZDRQ, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.851890448 (377h40m56.313982921s) [noise=true , address=77.90.31.10:4519 , piece_id=V2GSUV4F3UPHYDDMXLVVEH2D4J6VR34KYDQXG3INMSPURVBHKSQA, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.852126426 (377h40m56.314218909s) [noise=true , address=45.91.116.4:4938 , piece_id=FOJXDI5IXQCC376LXKJJDCBSRX2QH26DYDYDNTAHXJCG2A73HAAA, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.852459300 (377h40m56.314551784s) [noise=true , address=145.79.227.2:4114 , piece_id=7OCSXG2XFLWRKIQGNCGTKU5XDQ2ONJHVOZMHBN7XR2U26ADGVH6Q, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.852596652 (377h40m56.314689135s) [noise=true , address=45.91.116.6:4974 , piece_id=3AYAZL5SW2IGCUCRN37EQYTLIJ4NZBTDPF2MPY5JFBHWHNBZIOOQ, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.852980553 (377h40m56.315073026s) [noise=true , address=77.90.31.19:4841 , piece_id=ASN5OVUNTZCGGSAA5MUCS6MNJIX3FEZ2VW7FK5BMCTAONBXXY3FQ, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.853385755 (377h40m56.315478228s) [noise=true , address=77.90.31.11:4555 , piece_id=4MZYGHEZMRHHCW6Q3TMZU6SN23KYZBXEXSNHSAK6PHYDSWAWQ32Q, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.853728116 (377h40m56.315820589s) [noise=true , address=45.91.116.8:4047 , piece_id=VWTA33SJZ3XIX2PZTNLNQGFUFNVIEOOLZTERBZMBDWPNA37GHEZA, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:10.854074115 (377h40m56.316166628s) [noise=true , address=77.90.31.14:4661 , piece_id=CYXCRZJY7KBVOL2K5XKA6ZDCB5IXZANC6KTSXPY36EWFGAFNYZEQ, seg_pos={0 0}, upload=332286]: Uploading piece... Oct 9 07:12:12.484851589 (377h40m57.946944062s) [upload=332286]: Next returned nil segment Oct 9 07:12:12.484871416 (377h40m57.946963889s) [upload=332286]: Waiting for error group managing segments... Oct 9 07:12:12.872178437 (377h40m58.33427091s) [noise=true , address=145.79.227.12:4767 , piece_id=WMWS3EMQ2SDPXMKIL3MKYQ53F5Z2K4M62KXBAIX5I6WU45Z3PR4A, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:12.910941411 (377h40m58.373033894s) [noise=true , address=145.79.227.19:4079 , piece_id=U2OBNAMDOXJZ6TLDOBZTLMBZ5CQZIIVNGALVQQAYYWUAT5NB4NPA, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:12.913498009 (377h40m58.375590492s) [noise=true , address=145.79.227.5:4222 , piece_id=GTKYCUZMRS76ENMWX7HSPUHU7RJQWKP34RBW2N6WZCEJ7OYDDM7A, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:12.939828598 (377h40m58.401921081s) [noise=true , address=145.79.227.2:4114 , piece_id=7OCSXG2XFLWRKIQGNCGTKU5XDQ2ONJHVOZMHBN7XR2U26ADGVH6Q, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:12.941081394 (377h40m58.403173877s) [noise=true , address=145.79.227.18:4006 , piece_id=QAZMOFEQHLLXP43OSHIN5ZPYIKPHUAGJBO44DFLJ6H7QUU7GIGYA, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.036207666 (377h40m58.498300149s) [noise=true , address=45.91.116.12:4225 , piece_id=CDX7UWHLKYYC5F2VEH5PMMCSBMRKPYXNQZUD3IOOUIFOV6T7RMCQ, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.045059021 (377h40m58.507151494s) [noise=true , address=45.91.116.6:4974 , piece_id=3AYAZL5SW2IGCUCRN37EQYTLIJ4NZBTDPF2MPY5JFBHWHNBZIOOQ, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.047993470 (377h40m58.510085953s) [noise=true , address=77.90.31.11:4555 , piece_id=4MZYGHEZMRHHCW6Q3TMZU6SN23KYZBXEXSNHSAK6PHYDSWAWQ32Q, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.053001987 (377h40m58.51509447s) [noise=true , address=45.91.116.14:4189 , piece_id=KR46A5F2UWLK7VWDPVVL2RLYENHVY5GMA6HYBRQI7XY3MYLZZDRQ, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.066089959 (377h40m58.528182432s) [noise=true , address=77.90.31.14:4661 , piece_id=CYXCRZJY7KBVOL2K5XKA6ZDCB5IXZANC6KTSXPY36EWFGAFNYZEQ, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.072766432 (377h40m58.534858915s) [noise=true , address=45.91.116.5:4299 , piece_id=P3RENZ45B2IIQTFJYUNM6BP4A4CPPLTX3QXISNH3KVXUZQBQDV7Q, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.077083003 (377h40m58.539175486s) [noise=true , address=45.91.116.9:4263 , piece_id=7GXU2W3IIJRIDN5TXHRVIWALRLNGD5YLIH725AB6C2L34GN4ZCEQ, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.095378220 (377h40m58.557470733s) [noise=true , address=77.90.31.19:4841 , piece_id=ASN5OVUNTZCGGSAA5MUCS6MNJIX3FEZ2VW7FK5BMCTAONBXXY3FQ, seg_pos={0 0}, upload=332286]: Done uploading piece. err: <nil> Oct 9 07:12:13.095420682 (377h40m58.557513165s) [seg_pos={0 0}, upload=332286]: Segment reached optimal threshold of 13 pieces. Oct 9 07:12:13.095593229 (377h40m58.557685723s) [noise=true , address=45.91.116.8:4047 , piece_id=VWTA33SJZ3XIX2PZTNLNQGFUFNVIEOOLZTERBZMBDWPNA37GHEZA, seg_pos={0 0}, upload=332286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:13.095607617 (377h40m58.557700101s) [noise=true , address=45.91.116.8:4047 , piece_id=VWTA33SJZ3XIX2PZTNLNQGFUFNVIEOOLZTERBZMBDWPNA37GHEZA, seg_pos={0 0}, upload=332286]: Long tail cancellation due to optimal threshold Oct 9 07:12:13.095617436 (377h40m58.557709919s) [noise=true , address=45.91.116.4:4938 , piece_id=FOJXDI5IXQCC376LXKJJDCBSRX2QH26DYDYDNTAHXJCG2A73HAAA, seg_pos={0 0}, upload=332286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: write tcp 188.166.225.170:40720->45.91.116.4:4938: use of closed network connection; write tcp 188.166.225.170:40720->45.91.116.4:4938: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40720->45.91.116.4:4938: use of closed network connection Oct 9 07:12:13.095627274 (377h40m58.557719757s) [noise=true , address=45.91.116.4:4938 , piece_id=FOJXDI5IXQCC376LXKJJDCBSRX2QH26DYDYDNTAHXJCG2A73HAAA, seg_pos={0 0}, upload=332286]: Long tail cancellation due to optimal threshold Oct 9 07:12:13.095723778 (377h40m58.557816261s) [noise=true , address=77.90.31.10:4519 , piece_id=V2GSUV4F3UPHYDDMXLVVEH2D4J6VR34KYDQXG3INMSPURVBHKSQA, seg_pos={0 0}, upload=332286]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Oct 9 07:12:13.095733508 (377h40m58.557825981s) [noise=true , address=77.90.31.10:4519 , piece_id=V2GSUV4F3UPHYDDMXLVVEH2D4J6VR34KYDQXG3INMSPURVBHKSQA, seg_pos={0 0}, upload=332286]: Long tail cancellation due to optimal threshold Oct 9 07:12:13.095754176 (377h40m58.557846659s) [seg_pos={0 0}, upload=332286]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 9 07:12:13.095840571 (377h40m58.557933074s) [upload=332286]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 9 07:12:13.095847755 (377h40m58.557940228s) [upload=332286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 9 07:12:13.577083848 (377h40m59.039176321s) [upload=332286]: Done uploading segments... Oct 9 07:12:13.577106071 (377h40m59.039198554s) [upload=332286]: Upload finished. err: <nil> Duration: 30.002862891s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 4.828020877s FAIL (2025-10-08 23:19:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.828055293s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 4.82079574s FAIL (2025-10-08 23:18:50 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.820821731s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 4.91021955s FAIL (2025-10-08 23:17:52 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.910250267s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 4.787012581s FAIL (2025-10-08 23:17:36 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.787044843s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 4.781937758s FAIL (2025-10-08 23:16:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.781962595s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 4.829041563s FAIL (2025-10-08 23:16:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.829059847s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 4.840821593s FAIL (2025-10-08 23:15:49 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.840858706s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 4.820469848s FAIL (2025-10-08 23:15:16 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.820506147s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 4.910247385s FAIL (2025-10-08 23:14:05 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.910275929s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 4.771200193s FAIL (2025-10-08 23:13:28 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.771221513s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 4.762022566s FAIL (2025-10-08 23:12:49 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.762080638s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 4.796838812s FAIL (2025-10-08 23:12:25 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.796868296s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.01799332s FAIL (2025-10-08 23:11:27 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 5.018019852s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 4.785543365s FAIL (2025-10-08 23:10:51 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.875751988 (369h3m48.337844471s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876137592 (369h3m48.338230075s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876483491 (369h3m48.338575974s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.876876449 (369h3m48.338968942s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877230564 (369h3m48.339323047s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877528851 (369h3m48.339621334s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.877883356 (369h3m48.339975839s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878256327 (369h3m48.34034881s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878572879 (369h3m48.340665362s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:02.878874414 (369h3m48.340966907s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Uploading piece... Oct 8 22:35:03.737676059 (369h3m49.199768542s) [upload=324661]: Next returned nil segment Oct 8 22:35:03.737706437 (369h3m49.19979892s) [upload=324661]: Waiting for error group managing segments... Oct 8 22:35:04.154912905 (369h3m49.617005388s) [noise=true , address=89.149.201.147:4001 , piece_id=EACRU5HIJSIM4CZJRQROGLYAZAMEWX4MVTBZXELKZ6HQ3XIJ3VPQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.157015228 (369h3m49.619107711s) [noise=true , address=81.171.14.70:4045 , piece_id=2QSJJFUL54MGLENVQA5GCFEDJOVQF773NZVTAYRMFCC3CSVKL3SA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.165226393 (369h3m49.627318876s) [noise=true , address=81.171.14.70:4030 , piece_id=KZSHPCIUM3VWSLMMMD34WDMRRAO2ZVSGVLAZYHO3IF6WT33ITT7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.166862138 (369h3m49.628954621s) [noise=true , address=81.171.14.70:4036 , piece_id=O4A6TQ5VGNDW2YEW544DNWY4RQ2GEK7TUAJ5IODJP4CFAB7CGPNQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185143589 (369h3m49.647236072s) [noise=true , address=81.171.14.70:4031 , piece_id=D3SH7O7GT5Z27K7TVJEOH2WIZXOU5FHA5AY5HQF4EWJY3ZXWU5HA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.185385570 (369h3m49.647478043s) [noise=true , address=89.149.201.147:4022 , piece_id=6MSLRL2W6OD2WGZZGEVA2J6EIAYPDPDEGIRJ2U55IT6A6DJPUUFA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.211571884 (369h3m49.673664367s) [noise=true , address=89.149.201.147:4020 , piece_id=CDMZVSCSBZELN2MWEZQ2U4ZHUCTWWEP5OCZSGNKJ3VD76WEXMW2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.213521616 (369h3m49.67561411s) [noise=true , address=81.171.14.70:4044 , piece_id=F2O4KGC7GJZJYJTOPO72L4QDDCWUKYEDWEXUMY3Z4AIKMVSPPGCQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.595702097 (369h3m50.05779459s) [noise=true , address=89.149.201.147:4010 , piece_id=ZJERJO5G2SYR3YWFDF343XWOJPBTQKPIWNND7TS5JK4ZN5DLTDZA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.642071383 (369h3m50.104164057s) [noise=true , address=81.171.14.70:4047 , piece_id=X5YBB2PV2QSPTRPM7XJDFKA4Q4TQ3RAV47ZNIR4MQAAZ555DOGVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.643774677 (369h3m50.10586716s) [noise=true , address=81.171.14.70:4048 , piece_id=PD5LMLPNURZQQQMFCCR5HMDPJJB6BYNMZVKH57SCEIVPM7FGCR2A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.665648598 (369h3m50.127741081s) [noise=true , address=89.149.201.147:4006 , piece_id=RH5OKRZJITRYVG55LJTGAOITIF464U57LI7VBG5G73UE4JCHQG4A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.699317450 (369h3m50.161409933s) [noise=true , address=185.167.204.205:4014 , piece_id=ZFPAYUUKEQ7URS4MQP2WPQXYQTB2O66SNTLGZBWTATRK3DT4RAVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749079757 (369h3m50.21117224s) [noise=true , address=89.149.201.147:4021 , piece_id=YCYQ5W5VEOIK7LSHBW6LX63T5OQHXIYTCEFAQNCB3CIXPNQPVCNA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.749834623 (369h3m50.211927107s) [noise=true , address=185.167.204.205:4019 , piece_id=GRWC7PSUMXREB5VVGDQIXKZSMMHHNVS42SYSZYDTWW7BFQALUWGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.750011571 (369h3m50.212104054s) [noise=true , address=89.149.201.147:4007 , piece_id=A67SAWLWBP2V3MUFO7UJQMSDC5YVEZXRVFC6GDF4MRZZ4MTK2QTA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.753769266 (369h3m50.215861749s) [noise=true , address=185.167.204.205:4024 , piece_id=VDQXW2FU3STZVPDN7DOJJ6A5QYZU2LXERJDJNN2G74WQSAU3JXGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.765203619 (369h3m50.227296122s) [noise=true , address=185.167.204.206:4041 , piece_id=FQ2PVPI3NY4Z7K7HJ5OE7RXJI4YEKYY7WIYVQLWS4ZUY7MZGMEBA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.767935260 (369h3m50.230027743s) [noise=true , address=185.167.204.206:4038 , piece_id=5IZALR6RAAXELUQ7FIKKB7Z6ZTUQBMT2DUY7UTCL5RV2G2RVC3YQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.773313482 (369h3m50.235405965s) [noise=true , address=185.167.204.205:4034 , piece_id=RSI4KIGUP2BKETEWW5AWBKNMJOA53RDC6E3IDJ35VNJUNUEX2F3Q, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.778901051 (369h3m50.240993534s) [noise=true , address=185.167.204.206:4063 , piece_id=EURZTWZH5AY22QI5ZRKR4AIJQLRAOC6O3EETLZLQYELIW56MDCWQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.803093288 (369h3m50.265185771s) [noise=true , address=185.167.204.206:4052 , piece_id=ZKPYDVPEK7VD3YHO7PAQZB6BQNZN54YDILUJMVJ5ZNYUW7Y4SU7A, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.808927127 (369h3m50.27101962s) [noise=true , address=185.167.204.206:4059 , piece_id=E7DLF2RL3JIYVUI5X6ZUL7BLXOB4UCABJD35OQJS4E2HG7N7RMLA, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.809124583 (369h3m50.271217066s) [noise=true , address=185.167.204.206:4065 , piece_id=4Z4K7DAIKMPJODV3S4H6UGFENC2CVSFAOKP3E5XR5UBCSP7X3SVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.814617232 (369h3m50.276709715s) [noise=true , address=185.167.204.205:4036 , piece_id=BHEWHHTDNNPGQ2PP34T2R4KSMAC7YMOBTEEYNUZYOVJIO76V3DVQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816272294 (369h3m50.278364777s) [noise=true , address=185.167.204.205:4001 , piece_id=DVDYT4J6T3XITZ45EVH7P3T64DYEVLY47NPZ77ELNCX4IO77GDEQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: <nil> Oct 8 22:35:04.816306589 (369h3m50.278399082s) [seg_pos={0 0}, upload=324661]: Segment reached optimal threshold of 26 pieces. Oct 8 22:35:04.816489868 (369h3m50.278582351s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816500468 (369h3m50.278592951s) [noise=true , address=185.167.204.205:4013 , piece_id=66PUXZWOJ4AXKZT5HIRTUQ3EJDD65PMS7MIYY7RCDBVCKFRWOEGQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816583034 (369h3m50.278675518s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExKH4MVQ3r3VHXmBuTWSGAsqp9DyxxsojedEu96rtn4Jwk17u): protocol: expected piece hash; context canceled; EOF Oct 8 22:35:04.816593124 (369h3m50.278685617s) [noise=true , address=185.167.204.206:4054 , piece_id=TRAEDPNZ3V5E4FOC7VF6PITAIAS2WU3VKZ6FN75JFYSWMXGO4JOQ, seg_pos={0 0}, upload=324661]: Long tail cancellation due to optimal threshold Oct 8 22:35:04.816616959 (369h3m50.278709442s) [seg_pos={0 0}, upload=324661]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Oct 8 22:35:04.817060835 (369h3m50.279153338s) [upload=324661]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 8 22:35:04.817071896 (369h3m50.279164379s) [upload=324661]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 8 22:35:05.178046607 (369h3m50.64013908s) [upload=324661]: Done uploading segments... Oct 8 22:35:05.178071084 (369h3m50.640163557s) [upload=324661]: Upload finished. err: <nil> Duration: 4.785575456s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)