Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.838540864s
FAIL (2025-07-27 21:09:54 UTC)
orbiter warning: uplink: too many requests
Duration: 5.838568377s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...X.................X.............X........................] duration:
2.050646408s
FAIL (2025-07-27 21:09:50 UTC)
orbiter warning: uplink: too many requests
Duration: 2.05067381s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.904673225s
FAIL (2025-07-27 21:09:29 UTC)
orbiter warning: uplink: too many requests
Duration: 5.904709034s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....X.................X.............X.......................] duration:
2.090223119s
FAIL (2025-07-27 21:09:09 UTC)
orbiter warning: uplink: too many requests
Duration: 2.090252315s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
6.033454108s
FAIL (2025-07-27 21:08:55 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 6.033479076s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.....X.................X.............X......................] duration:
1.8556329s
FAIL (2025-07-27 21:08:40 UTC)
orbiter warning: uplink: too many requests
Duration: 1.855664621s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.772213673s
FAIL (2025-07-27 21:08:32 UTC)
orbiter warning: uplink: too many requests
Duration: 5.772241085s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.724335719s
FAIL (2025-07-27 21:08:18 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.724364254s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [......X.................X.............X.....................] duration:
1.920925285s
FAIL (2025-07-27 21:08:04 UTC)
orbiter warning: uplink: too many requests
Duration: 1.920953951s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.......X.................X.............X....................] duration:
1.907815111s
FAIL (2025-07-27 21:07:35 UTC)
orbiter warning: uplink: too many requests
Duration: 1.90784107s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.893568088s
FAIL (2025-07-27 21:07:19 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.893591582s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.861707759s
FAIL (2025-07-27 21:07:17 UTC)
orbiter warning: uplink: too many requests
Duration: 5.861729753s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [........X.................X.............X...................] duration:
1.808709423s
FAIL (2025-07-27 21:06:50 UTC)
orbiter warning: uplink: too many requests
Duration: 1.808737237s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.798687014s
FAIL (2025-07-27 21:06:26 UTC)
orbiter warning: uplink: too many requests
Duration: 5.798713785s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.806321961s
FAIL (2025-07-27 21:06:12 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.806341628s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.........X.................X.............X..................] duration:
1.873890495s
FAIL (2025-07-27 21:06:02 UTC)
orbiter warning: uplink: too many requests
Duration: 1.873916704s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.846386396s
FAIL (2025-07-27 21:05:50 UTC)
orbiter warning: uplink: too many requests
Duration: 5.846406875s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.719033434s
FAIL (2025-07-27 21:05:48 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.719063721s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..........X.................X.............X.................] duration:
1.831747352s
FAIL (2025-07-27 21:05:25 UTC)
orbiter warning: uplink: too many requests
Duration: 1.83177186s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.757844109s
FAIL (2025-07-27 21:05:20 UTC)
orbiter warning: uplink: too many requests
Duration: 5.757866672s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
6.355879491s
FAIL (2025-07-27 21:05:17 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 6.355918454s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.920869443s
FAIL (2025-07-27 21:04:58 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.920893147s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...........X.................X.............X................] duration:
1.854616487s
FAIL (2025-07-27 21:04:33 UTC)
orbiter warning: uplink: too many requests
Duration: 1.854645212s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.783223017s
FAIL (2025-07-27 21:04:22 UTC)
orbiter warning: uplink: too many requests
Duration: 5.78324062s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.878340181s
FAIL (2025-07-27 21:04:13 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.878371631s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [............X.................X.............X...............] duration:
1.883860871s
FAIL (2025-07-27 21:04:11 UTC)
orbiter warning: uplink: too many requests
Duration: 1.883890968s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.894857295s
FAIL (2025-07-27 21:03:53 UTC)
orbiter warning: uplink: too many requests
Duration: 5.894890809s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.............X.................X.............X..............] duration:
1.841651702s
FAIL (2025-07-27 21:03:41 UTC)
orbiter warning: uplink: too many requests
Duration: 1.841718158s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.08090741s
FAIL (2025-07-27 21:03:31 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 6.080944812s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.161553728s
FAIL (2025-07-27 21:03:19 UTC)
orbiter warning: uplink: too many requests
Duration: 6.161580379s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.842805867s
FAIL (2025-07-27 21:03:14 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.842832628s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..............X.................X.............X.............] duration:
1.941905194s
FAIL (2025-07-27 21:03:01 UTC)
orbiter warning: uplink: too many requests
Duration: 1.94193433s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.924324965s
FAIL (2025-07-27 21:02:45 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.924370751s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.887143154s
FAIL (2025-07-27 21:02:43 UTC)
orbiter warning: uplink: too many requests
Duration: 5.887170026s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...............X.................X.............X............] duration:
1.964113085s
FAIL (2025-07-27 21:02:43 UTC)
orbiter warning: uplink: too many requests
Duration: 1.964137071s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [................X.................X.............X...........] duration:
1.833737441s
FAIL (2025-07-27 21:02:22 UTC)
orbiter warning: uplink: too many requests
Duration: 1.833781846s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.7178302s
FAIL (2025-07-27 21:02:03 UTC)
orbiter warning: uplink: too many requests
Duration: 5.71786169s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.808201626s
FAIL (2025-07-27 21:01:57 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.808221263s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.................X.................X.............X..........] duration:
1.863678728s
FAIL (2025-07-27 21:01:29 UTC)
orbiter warning: uplink: too many requests
Duration: 1.863704767s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.802803408s
FAIL (2025-07-27 21:01:22 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.802837143s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.806767841s
FAIL (2025-07-27 21:00:59 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.806795082s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
6.166741452s
FAIL (2025-07-27 21:00:58 UTC)
orbiter warning: uplink: too many requests
Duration: 6.166778984s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..................X.................X.............X.........] duration:
2.041389437s
FAIL (2025-07-27 21:00:55 UTC)
orbiter warning: uplink: too many requests
Duration: 2.041418372s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.694287668s
FAIL (2025-07-27 21:00:28 UTC)
orbiter warning: uplink: too many requests
Duration: 5.694309369s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...................X.................X.............X........] duration:
1.848737168s
FAIL (2025-07-27 21:00:28 UTC)
orbiter warning: uplink: too many requests
Duration: 1.848768349s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.975348957s
FAIL (2025-07-27 21:00:13 UTC)
orbiter warning: uplink: too many requests
Duration: 5.975382121s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....................X.................X.............X.......] duration:
1.852505499s
FAIL (2025-07-27 20:59:59 UTC)
orbiter warning: uplink: too many requests
Duration: 1.852531979s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
5.77279318s
FAIL (2025-07-27 20:59:55 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.772822246s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.782064098s
FAIL (2025-07-27 20:59:39 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.782092533s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.801470308s
FAIL (2025-07-27 20:59:35 UTC)
orbiter warning: uplink: too many requests
Duration: 5.801495045s
core
uplink
us1
vivint
|