Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X........................................................X] duration:
4.252658173s
FAIL (2025-07-27 21:15:40 UTC)
orbiter warning: uplink: object not found ("orbiter_us_0/4M")
Duration: 4.252687128s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............X.............X...............................] duration:
1.882757072s
FAIL (2025-07-27 21:15:40 UTC)
orbiter warning: uplink: too many requests
Duration: 1.882797481s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
6.624871661s
FAIL (2025-07-27 21:15:23 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 6.624907359s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............X.............X..............................] duration:
1.817544691s
FAIL (2025-07-27 21:14:50 UTC)
orbiter warning: uplink: too many requests
Duration: 1.817574439s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
5.837142128s
FAIL (2025-07-27 21:14:35 UTC)
orbiter warning: uplink: too many requests
Duration: 5.837171605s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.838690582s
FAIL (2025-07-27 21:14:22 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.838714638s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................X.............X.............................] duration:
1.915800642s
FAIL (2025-07-27 21:13:52 UTC)
orbiter warning: uplink: too many requests
Duration: 1.91583066s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.760935003s
FAIL (2025-07-27 21:13:36 UTC)
orbiter warning: uplink: too many requests
Duration: 5.760979187s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.814523843s
FAIL (2025-07-27 21:13:28 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.814552389s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
5.756303841s
FAIL (2025-07-27 21:13:21 UTC)
orbiter warning: uplink: too many requests
Duration: 5.756329901s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................X.............X............................] duration:
1.987827362s
FAIL (2025-07-27 21:12:56 UTC)
orbiter warning: uplink: too many requests
Duration: 1.987869482s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.937000886s
FAIL (2025-07-27 21:12:27 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.937026996s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
5.736946343s
FAIL (2025-07-27 21:12:10 UTC)
orbiter warning: uplink: too many requests
Duration: 5.736999545s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [X.................X.............X...........................] duration:
1.860138853s
FAIL (2025-07-27 21:11:48 UTC)
orbiter warning: uplink: too many requests
Duration: 1.860165003s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
5.897067454s
FAIL (2025-07-27 21:11:41 UTC)
orbiter warning: uplink: too many requests
Duration: 5.897103394s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.849846138s
FAIL (2025-07-27 21:11:31 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.849875094s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.X.................X.............X..........................] duration:
1.849226815s
FAIL (2025-07-27 21:11:11 UTC)
orbiter warning: uplink: too many requests
Duration: 1.849257093s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
5.703996854s
FAIL (2025-07-27 21:11:08 UTC)
orbiter warning: uplink: too many requests
Duration: 5.704092416s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
5.894783273s
FAIL (2025-07-27 21:10:58 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.894809232s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
5.806399478s
FAIL (2025-07-27 21:10:42 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.806426551s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..X.................X.............X.........................] duration:
1.888602382s
FAIL (2025-07-27 21:10:36 UTC)
orbiter warning: uplink: too many requests
Duration: 1.888629473s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.889751575s
FAIL (2025-07-27 21:10:07 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.889771552s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.838540864s
FAIL (2025-07-27 21:09:54 UTC)
orbiter warning: uplink: too many requests
Duration: 5.838568377s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...X.................X.............X........................] duration:
2.050646408s
FAIL (2025-07-27 21:09:50 UTC)
orbiter warning: uplink: too many requests
Duration: 2.05067381s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.904673225s
FAIL (2025-07-27 21:09:29 UTC)
orbiter warning: uplink: too many requests
Duration: 5.904709034s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....X.................X.............X.......................] duration:
2.090223119s
FAIL (2025-07-27 21:09:09 UTC)
orbiter warning: uplink: too many requests
Duration: 2.090252315s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
6.033454108s
FAIL (2025-07-27 21:08:55 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 6.033479076s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.....X.................X.............X......................] duration:
1.8556329s
FAIL (2025-07-27 21:08:40 UTC)
orbiter warning: uplink: too many requests
Duration: 1.855664621s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.772213673s
FAIL (2025-07-27 21:08:32 UTC)
orbiter warning: uplink: too many requests
Duration: 5.772241085s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.724335719s
FAIL (2025-07-27 21:08:18 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.724364254s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [......X.................X.............X.....................] duration:
1.920925285s
FAIL (2025-07-27 21:08:04 UTC)
orbiter warning: uplink: too many requests
Duration: 1.920953951s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.......X.................X.............X....................] duration:
1.907815111s
FAIL (2025-07-27 21:07:35 UTC)
orbiter warning: uplink: too many requests
Duration: 1.90784107s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.893568088s
FAIL (2025-07-27 21:07:19 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.893591582s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.861707759s
FAIL (2025-07-27 21:07:17 UTC)
orbiter warning: uplink: too many requests
Duration: 5.861729753s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [........X.................X.............X...................] duration:
1.808709423s
FAIL (2025-07-27 21:06:50 UTC)
orbiter warning: uplink: too many requests
Duration: 1.808737237s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.798687014s
FAIL (2025-07-27 21:06:26 UTC)
orbiter warning: uplink: too many requests
Duration: 5.798713785s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.806321961s
FAIL (2025-07-27 21:06:12 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.806341628s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.........X.................X.............X..................] duration:
1.873890495s
FAIL (2025-07-27 21:06:02 UTC)
orbiter warning: uplink: too many requests
Duration: 1.873916704s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.846386396s
FAIL (2025-07-27 21:05:50 UTC)
orbiter warning: uplink: too many requests
Duration: 5.846406875s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.719033434s
FAIL (2025-07-27 21:05:48 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.719063721s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..........X.................X.............X.................] duration:
1.831747352s
FAIL (2025-07-27 21:05:25 UTC)
orbiter warning: uplink: too many requests
Duration: 1.83177186s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.757844109s
FAIL (2025-07-27 21:05:20 UTC)
orbiter warning: uplink: too many requests
Duration: 5.757866672s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
6.355879491s
FAIL (2025-07-27 21:05:17 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 6.355918454s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.920869443s
FAIL (2025-07-27 21:04:58 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.920893147s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...........X.................X.............X................] duration:
1.854616487s
FAIL (2025-07-27 21:04:33 UTC)
orbiter warning: uplink: too many requests
Duration: 1.854645212s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.783223017s
FAIL (2025-07-27 21:04:22 UTC)
orbiter warning: uplink: too many requests
Duration: 5.78324062s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.878340181s
FAIL (2025-07-27 21:04:13 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
NBYMF2YQ, seg_pos={0 0}, upload=78056]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34898->104.36.82.7:4909: use of closed network connection
Jul 27 20:55:33.507192913 (81h19m43.229755953s) [seg_pos={0 0}, upload=78056]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:55:33.507272095 (81h19m43.229835135s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:55:33.507278908 (81h19m43.229841948s) [upload=78056]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:55:33.873750499 (81h19m43.596313549s) [upload=78056]: Done uploading segments...
Jul 27 20:55:33.873771960 (81h19m43.59633502s) [upload=78056]: Upload finished. err: <nil>
Jul 27 20:56:44.063819950 (81h20m53.78638298s) [upload=78076]: Starting upload
Jul 27 20:56:44.063870106 (81h20m53.786433146s) [upload=78076]: Done starting upload
Jul 27 20:56:44.064058887 (81h20m53.786621927s) [upload=78076]: Uploading segments...
Jul 27 20:56:44.064105396 (81h20m53.786668436s) [upload=78076]: Got next segment. Inline: false
Jul 27 20:56:44.064136426 (81h20m53.786699466s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 27 20:56:44.064301271 (81h20m53.786864321s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 27 20:56:44.584553631 (81h20m54.307116671s) [seg_pos={0 0}, upload=78076]: Begin upload segment...
Jul 27 20:56:44.584612734 (81h20m54.307175784s) [seg_pos={0 0}, upload=78076]: Done begin upload segment.
Jul 27 20:56:44.584637962 (81h20m54.307201002s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.584670444 (81h20m54.307233484s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585060231 (81h20m54.307623271s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585139933 (81h20m54.307702963s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585361397 (81h20m54.307924437s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585453122 (81h20m54.308016162s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585646031 (81h20m54.308209081s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585756052 (81h20m54.308319092s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.585991612 (81h20m54.308554652s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586051648 (81h20m54.308614688s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586298259 (81h20m54.30886129s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586318758 (81h20m54.308881798s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586610727 (81h20m54.309173777s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.586908527 (81h20m54.309471567s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587209813 (81h20m54.309772853s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:44.587534054 (81h20m54.310097094s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Uploading piece...
Jul 27 20:56:46.339251587 (81h20m56.061814627s) [upload=78076]: Next returned nil segment
Jul 27 20:56:46.339268218 (81h20m56.061831248s) [upload=78076]: Waiting for error group managing segments...
Jul 27 20:56:46.616169719 (81h20m56.338732749s) [noise=true , address=45.91.116.10:4081 , piece_id=5OBJPHS7YVSLQ7R25I6B33XXTS5IYKF4RLAUGRORWNVF2VIJZOKA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.672944635 (81h20m56.395507676s) [noise=true , address=77.90.31.12:4590 , piece_id=RY3N4AP2KLW2RGQDSYTURIEAW7FJKDRODVBJJ574AGWREXJOIXOA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.679116684 (81h20m56.401679724s) [noise=true , address=77.90.31.14:4663 , piece_id=VXGT4HSKJ6LU4CNLPXVOD64RFMTIXHSOIHP3FYKTJEUREGCXJHPA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.689698961 (81h20m56.412262001s) [noise=true , address=77.90.31.16:4733 , piece_id=UUS54WDRK744TKZLXXAAPRBPCM2OUGPWCRNV57FUZ4K2KXMVG23Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.691556325 (81h20m56.414119365s) [noise=true , address=77.90.31.15:4697 , piece_id=5DDY52MEVML4BBMTJFPJD3SWD4UYLYQ4WT5DGBPIQWBIXMPTCHRQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.696368232 (81h20m56.418931272s) [noise=true , address=45.91.116.9:4262 , piece_id=7NM6FTVFO4RO7ZUI2A6ONJSQHSKTBNUJTE4RE2PG73V7SDIDFTLQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.701404117 (81h20m56.423967157s) [noise=true , address=77.90.31.13:4628 , piece_id=XMP7M7ILT5U7UXGYSM4UB5VBMWUHIDRGTZPLZRYLK3W2NTM4WEDQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.708417285 (81h20m56.430980325s) [noise=true , address=77.90.31.15:4699 , piece_id=AL7HA5TX46M3ZF2BOZGNGGFJ5IHHSDRT4YHUN2XEL2XPEMLISRWQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.711882846 (81h20m56.434445886s) [noise=true , address=45.91.116.14:4190 , piece_id=G4FLK67VLPGRJJUNECETUKGYS3OMQA4CGBKCWPDHVHT632JUHMBQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.730484234 (81h20m56.453047274s) [noise=true , address=45.91.116.8:4047 , piece_id=VTO4NXZJKRF4SDOLNBTOUC4T6LD7UGCHJQTJOGGS2LPF7CPZWGIA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.733559909 (81h20m56.456122949s) [noise=true , address=77.90.31.16:4734 , piece_id=GJJTLE73ATBF2FSCY7QOGWXORADSKXLJXNGDH34DZTSPPTQKSXJA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.761219480 (81h20m56.48378252s) [noise=true , address=45.91.116.11:4118 , piece_id=WAANSRE3AOQ74X575MUJEMFZVVSGECFE5IXRID6UFB6LMK76G5VQ, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764041981 (81h20m56.486605021s) [noise=true , address=77.90.31.10:4520 , piece_id=XECJATSD772ZU45XV2KDWE67C2MQHRH7J2MLTCPWHZLWA7YKF36Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: <nil>
Jul 27 20:56:46.764051749 (81h20m56.486614789s) [seg_pos={0 0}, upload=78076]: Segment reached optimal threshold of 13 pieces.
Jul 27 20:56:46.764265288 (81h20m56.486828328s) [noise=true , address=45.91.116.7:4011 , piece_id=4MZ6Y2VDJEYHGHOM7QIO4OB5OF3EYPXCACP5UR7V2BCZN7PN4T4Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764492743 (81h20m56.487055783s) [noise=true , address=45.91.116.7:4010 , piece_id=QJCPKYGZ2MJGMIZGDXIDQDZQMEE2HUBWXZ7KRAGIJD564MOQWX2Q, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764577445 (81h20m56.487140485s) [noise=true , address=77.90.31.18:4806 , piece_id=PECBEVTQRQJUPRHUS2FDKQHQF4EAAXW226HVFYM5EGCF72ZKRXEA, seg_pos={0 0}, upload=78076]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF
Jul 27 20:56:46.764592364 (81h20m56.487155404s) [seg_pos={0 0}, upload=78076]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 27 20:56:46.764671274 (81h20m56.487234314s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 27 20:56:46.764675643 (81h20m56.487238673s) [upload=78076]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 27 20:56:47.043439135 (81h20m56.766002255s) [upload=78076]: Done uploading segments...
Jul 27 20:56:47.043463021 (81h20m56.766026061s) [upload=78076]: Upload finished. err: <nil>
Duration: 5.878371631s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [............X.................X.............X...............] duration:
1.883860871s
FAIL (2025-07-27 21:04:11 UTC)
orbiter warning: uplink: too many requests
Duration: 1.883890968s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.894857295s
FAIL (2025-07-27 21:03:53 UTC)
orbiter warning: uplink: too many requests
Duration: 5.894890809s
core
uplink
us1
vivint
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.............X.................X.............X..............] duration:
1.841651702s
FAIL (2025-07-27 21:03:41 UTC)
orbiter warning: uplink: too many requests
Duration: 1.841718158s
core
uplink
us1
vivint
|