Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.345275819s FAIL (2025-09-09 09:31:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.345305115s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X........X..................................................] duration: 7.691032879s FAIL (2025-09-09 09:30:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 7.691077825s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.226582506s FAIL (2025-09-09 09:30:10 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.226614779s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X........X.................................................] duration: 10.274706607s FAIL (2025-09-09 09:29:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 10.274735272s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.123366447s FAIL (2025-09-09 09:29:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.123392578s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.184353957s FAIL (2025-09-09 09:28:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.184382651s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X........X................................................] duration: 12.18339265s FAIL (2025-09-09 09:28:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 12.183418639s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X........X...............................................] duration: 9.848293437s FAIL (2025-09-09 09:28:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 9.848320119s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.238941749s FAIL (2025-09-09 09:28:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.238970192s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X........X..............................................] duration: 12.948125082s FAIL (2025-09-09 09:27:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 12.948153527s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.300517676s FAIL (2025-09-09 09:27:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.300563734s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X........X.............................................] duration: 8.260633491s FAIL (2025-09-09 09:27:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 8.260664259s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.142707221s FAIL (2025-09-09 09:27:05 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.142736566s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.457810453s FAIL (2025-09-09 09:26:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.457841823s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X........X............................................] duration: 6.13085178s FAIL (2025-09-09 09:26:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 6.130877459s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.768143771s FAIL (2025-09-09 09:25:32 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.768170993s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X........X...........................................] duration: 8.519199922s FAIL (2025-09-09 09:25:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 8.519233236s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.23254635s FAIL (2025-09-09 09:25:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.232574864s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X........X..........................................] duration: 7.006277082s FAIL (2025-09-09 09:24:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 7.006307992s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.405956571s FAIL (2025-09-09 09:24:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.405994483s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X........X.........................................] duration: 10.614191619s FAIL (2025-09-09 09:23:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 10.614219433s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.506695705s FAIL (2025-09-09 09:23:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.506720593s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X........X........................................] duration: 6.079817449s FAIL (2025-09-09 09:22:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 6.079853458s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.178922506s FAIL (2025-09-09 09:22:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.178949138s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X........X.......................................] duration: 7.235638867s FAIL (2025-09-09 09:22:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 7.235668222s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.300903654s FAIL (2025-09-09 09:21:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.301056717s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X........X......................................] duration: 12.275155055s FAIL (2025-09-09 09:21:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 12.27518456s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X........X.....................................] duration: 13.166548119s FAIL (2025-09-09 09:20:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 13.16657459s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.309985845s FAIL (2025-09-09 09:20:27 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.310024658s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.206299404s FAIL (2025-09-09 09:19:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.206323219s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X........X....................................] duration: 6.722049923s FAIL (2025-09-09 09:19:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 6.72208021s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.454095382s FAIL (2025-09-09 09:18:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.454123226s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X........X...................................] duration: 6.118225827s FAIL (2025-09-09 09:18:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 6.118257748s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.366257387s FAIL (2025-09-09 09:18:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.366287576s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 7.039097144s FAIL (2025-09-09 09:17:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 7.039124376s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X........X..................................] duration: 6.396904257s FAIL (2025-09-09 09:17:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 6.396935596s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.31947913s FAIL (2025-09-09 09:17:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.319512374s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X........X.................................] duration: 6.819201909s FAIL (2025-09-09 09:17:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 6.819241955s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.539535208s FAIL (2025-09-09 09:17:01 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.539564432s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X........X................................] duration: 6.044090398s FAIL (2025-09-09 09:16:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 6.044110576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.107902725s FAIL (2025-09-09 09:16:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.107946708s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X........X...............................] duration: 8.379009998s FAIL (2025-09-09 09:15:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 8.379039564s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.396381562s FAIL (2025-09-09 09:15:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.396421269s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X........X..............................] duration: 7.066967415s FAIL (2025-09-09 09:15:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 7.06700106s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.667834506s FAIL (2025-09-09 09:14:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.667864655s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.211639259s FAIL (2025-09-09 09:14:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.211666861s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X........X.............................] duration: 5.983892916s FAIL (2025-09-09 09:14:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 5.983920339s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.313612158s FAIL (2025-09-09 09:13:31 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.313646943s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.428263578s FAIL (2025-09-09 09:12:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.428291261s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X........X............................] duration: 8.336572051s FAIL (2025-09-09 09:12:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 9 08:48:37.603346736 (67h20m13.642919846s) [upload=59256]: Starting upload Sep 9 08:48:37.603419805 (67h20m13.642992925s) [upload=59256]: Done starting upload Sep 9 08:48:37.603560626 (67h20m13.643133746s) [upload=59256]: Uploading segments... Sep 9 08:48:37.603658763 (67h20m13.643231883s) [upload=59256]: Got next segment. Inline: false Sep 9 08:48:37.603709961 (67h20m13.643283081s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 9 08:48:37.603714890 (67h20m13.64328801s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 9 08:48:37.986127806 (67h20m14.025700926s) [seg_pos={0 0}, upload=59256]: Begin upload segment... Sep 9 08:48:37.986183582 (67h20m14.025756702s) [seg_pos={0 0}, upload=59256]: Done begin upload segment. Sep 9 08:48:37.986225984 (67h20m14.025799104s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986237676 (67h20m14.025810796s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986694540 (67h20m14.02626765s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.986760556 (67h20m14.026333677s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987415671 (67h20m14.026988821s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987669677 (67h20m14.027242797s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.987794977 (67h20m14.027368097s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988122874 (67h20m14.027695994s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988373514 (67h20m14.027946634s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988546344 (67h20m14.028119464s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988652898 (67h20m14.028226018s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.988886315 (67h20m14.028459435s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989175137 (67h20m14.028748257s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989417802 (67h20m14.028990922s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989681587 (67h20m14.029254707s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:37.989995829 (67h20m14.029568949s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Uploading piece... Sep 9 08:48:39.727499450 (67h20m15.76707257s) [upload=59256]: Next returned nil segment Sep 9 08:48:39.727520830 (67h20m15.76709395s) [upload=59256]: Waiting for error group managing segments... Sep 9 08:48:40.014701339 (67h20m16.054274459s) [noise=true , address=145.79.227.5:4223 , piece_id=AQUNEJGOW5IRVALCKHWGUP6MQMSVWVRO743BIB6MMO2O4NMQY7QQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.072180223 (67h20m16.111753343s) [noise=true , address=145.79.227.6:4257 , piece_id=J623V6NZ3K2ME32GU5NH5BF5N2KALWQP3FSWGIBPAPHZWGK6Z5XQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.109365648 (67h20m16.148938768s) [noise=true , address=145.79.227.15:4874 , piece_id=O2URXXTO5MOQMMAFY2PJDRWZ62QSLEUIF6AY4FY46VQYR7DWQ62A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159800050 (67h20m16.19937316s) [noise=true , address=45.91.116.13:4155 , piece_id=MPMI3FZ6CKRFTT4IFLVD33KR2KV54JEINHK3X4OHJAFPBOHMIDOQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.159849555 (67h20m16.199422675s) [noise=true , address=77.90.31.13:4626 , piece_id=W2VVUE6HAGIXZLVKYTTFU6M4HF5NY6AKCIQHLYC7NK47PJWMCW4Q, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.161483943 (67h20m16.201057063s) [noise=true , address=77.90.31.21:4877 , piece_id=7M72M4RDZJ4M4LWRJ4U4JGJ5ATZP7LU33VP6Q3JBH27JT5NDHPSA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.167070705 (67h20m16.206643825s) [noise=true , address=145.79.227.16:4909 , piece_id=I446II33HHFVB6UGF47TCEJYHKBY2PGEFDX4M43QYTTH5NXSQEXQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.232702602 (67h20m16.272275722s) [noise=true , address=45.91.116.4:4937 , piece_id=5MIQOHSRDKT6UIJWNH2CCAQBAA4OEWKVZVYJYOEBR3VOANVCZO4A, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.242044851 (67h20m16.281617971s) [noise=true , address=77.90.31.18:4806 , piece_id=WVEPZ45N7VFOSZ72ANWCMJ5TVGNZIASG4C2QZMZO2AQYTGO2BGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.248682735 (67h20m16.288255855s) [noise=true , address=77.90.31.21:4878 , piece_id=HWA7W32FWRM77J7FMUGV3EN7XXV2NROBG6SAPJN6BNFHOAOPPPDQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.257725040 (67h20m16.29729815s) [noise=true , address=45.91.116.14:4191 , piece_id=NKITGRVIF77VKSGCXMVRTPZG63CY3NK3DIAFPY7VMBUPA6HNSXEA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.259052331 (67h20m16.298625451s) [noise=true , address=77.90.31.10:4517 , piece_id=SBTLRDAD63V3ZMUYD7CARX4M6VVCMNZJTPOXBKGBHSIEOXGJ6VAQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266110619 (67h20m16.305683739s) [noise=true , address=45.91.116.11:4119 , piece_id=N5QUIECOJ5RKXFK3VO44Z2ACS34A52QQ3TMJXA5VOQGPORNUBIIA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: <nil> Sep 9 08:48:40.266137240 (67h20m16.30571036s) [seg_pos={0 0}, upload=59256]: Segment reached optimal threshold of 13 pieces. Sep 9 08:48:40.266257359 (67h20m16.305830479s) [noise=true , address=45.91.116.11:4118 , piece_id=PWHCNFFDI7O234EO6NCWXYJQG3JPS7DFC7OI4HZ7X224T4LJBGTQ, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266314398 (67h20m16.305887518s) [noise=true , address=45.91.116.6:4974 , piece_id=ZBXZZDMZKQV43RM5SVEM42SHUOHY2MG2XMCSPVBTWLNKGMHVX4LA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266370075 (67h20m16.305943195s) [noise=true , address=77.90.31.12:4590 , piece_id=CYCSONSAYGQ2VLGHP63BIRMTFDRM4W4W6KYTTS2YJLW5WHA75EAA, seg_pos={0 0}, upload=59256]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Sep 9 08:48:40.266471809 (67h20m16.30604495s) [seg_pos={0 0}, upload=59256]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 9 08:48:40.266560709 (67h20m16.306133829s) [upload=59256]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 9 08:48:40.266565328 (67h20m16.306138448s) [upload=59256]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 9 08:48:40.878820307 (67h20m16.918393437s) [upload=59256]: Done uploading segments... Sep 9 08:48:40.878842610 (67h20m16.91841573s) [upload=59256]: Upload finished. err: <nil> Duration: 8.3366034s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)