Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [................X...............................X........X..] duration: 1.909277273s FAIL (2025-08-11 16:47:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 7ca3b38f59bef6700c47a71c0ce137786b019861ef4e351f330d4f1e41b86704 != before b8ccbd272fe59726b04686e0542c1e0964e89942b662558e091433dc0e7e3dc5 ======LOGS====== Aug 11 16:46:39.434150567 (71h20m33.865230698s) [upload=74358]: Starting upload Aug 11 16:46:39.434202827 (71h20m33.865282958s) [upload=74358]: Done starting upload Aug 11 16:46:39.434253474 (71h20m33.865333605s) [upload=74358]: Uploading segments... Aug 11 16:46:39.434289643 (71h20m33.865369774s) [upload=74358]: Got next segment. Inline: true Aug 11 16:46:39.434293440 (71h20m33.865373561s) [upload=74358]: Waiting for error group managing segments... Aug 11 16:46:39.434326714 (71h20m33.865406845s) [upload=74358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:46:39.434330481 (71h20m33.865410612s) [upload=74358]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:46:39.434332896 (71h20m33.865413017s) [upload=74358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:46:39.727280447 (71h20m34.158360588s) [upload=74358]: Done uploading segments... Aug 11 16:46:39.727301838 (71h20m34.158381969s) [upload=74358]: Upload finished. err: <nil> Duration:* 1.909308153s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.932431378s FAIL (2025-08-11 16:47:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.932459032s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.924473164s FAIL (2025-08-11 16:47:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.924501319s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................X...............................X........X.] duration: 1.837781874s FAIL (2025-08-11 16:47:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 7ca3b38f59bef6700c47a71c0ce137786b019861ef4e351f330d4f1e41b86704 != before b8ccbd272fe59726b04686e0542c1e0964e89942b662558e091433dc0e7e3dc5 ======LOGS====== Aug 11 16:46:39.434150567 (71h20m33.865230698s) [upload=74358]: Starting upload Aug 11 16:46:39.434202827 (71h20m33.865282958s) [upload=74358]: Done starting upload Aug 11 16:46:39.434253474 (71h20m33.865333605s) [upload=74358]: Uploading segments... Aug 11 16:46:39.434289643 (71h20m33.865369774s) [upload=74358]: Got next segment. Inline: true Aug 11 16:46:39.434293440 (71h20m33.865373561s) [upload=74358]: Waiting for error group managing segments... Aug 11 16:46:39.434326714 (71h20m33.865406845s) [upload=74358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:46:39.434330481 (71h20m33.865410612s) [upload=74358]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:46:39.434332896 (71h20m33.865413017s) [upload=74358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:46:39.727280447 (71h20m34.158360588s) [upload=74358]: Done uploading segments... Aug 11 16:46:39.727301838 (71h20m34.158381969s) [upload=74358]: Upload finished. err: <nil> Duration:* 1.837821029s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.842466126s FAIL (2025-08-11 16:46:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.842489761s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................X...............................X........X] duration: 1.728417035s FAIL (2025-08-11 16:46:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 7ca3b38f59bef6700c47a71c0ce137786b019861ef4e351f330d4f1e41b86704 != before b8ccbd272fe59726b04686e0542c1e0964e89942b662558e091433dc0e7e3dc5 ======LOGS====== Aug 11 16:46:39.434150567 (71h20m33.865230698s) [upload=74358]: Starting upload Aug 11 16:46:39.434202827 (71h20m33.865282958s) [upload=74358]: Done starting upload Aug 11 16:46:39.434253474 (71h20m33.865333605s) [upload=74358]: Uploading segments... Aug 11 16:46:39.434289643 (71h20m33.865369774s) [upload=74358]: Got next segment. Inline: true Aug 11 16:46:39.434293440 (71h20m33.865373561s) [upload=74358]: Waiting for error group managing segments... Aug 11 16:46:39.434326714 (71h20m33.865406845s) [upload=74358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:46:39.434330481 (71h20m33.865410612s) [upload=74358]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:46:39.434332896 (71h20m33.865413017s) [upload=74358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:46:39.727280447 (71h20m34.158360588s) [upload=74358]: Done uploading segments... Aug 11 16:46:39.727301838 (71h20m34.158381969s) [upload=74358]: Upload finished. err: <nil> Duration:* 1.72843959s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.026733119s FAIL (2025-08-11 16:46:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 6.026757186s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X............................................X..............] duration: 5.901869966s FAIL (2025-08-11 16:45:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.901893351s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.147103577s FAIL (2025-08-11 16:45:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 6.147131652s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................X...............................X........] duration: 1.909891976s FAIL (2025-08-11 16:45:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 1.909918866s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X............................................X.............] duration: 5.780315697s FAIL (2025-08-11 16:45:18 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.780344431s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................X...............................X.......] duration: 1.844234851s FAIL (2025-08-11 16:45:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 1.844267063s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X............................................X............] duration: 5.840344535s FAIL (2025-08-11 16:44:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.840369282s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................X...............................X......] duration: 1.885064294s FAIL (2025-08-11 16:44:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 1.885105292s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.042982468s FAIL (2025-08-11 16:44:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 6.043010783s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.988859023s FAIL (2025-08-11 16:43:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.988889311s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................X...............................X.....] duration: 1.828097393s FAIL (2025-08-11 16:43:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 1.828113854s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.96798977s FAIL (2025-08-11 16:43:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.968022804s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X............................................X...........] duration: 5.884472569s FAIL (2025-08-11 16:43:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.884500212s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X............................................X..........] duration: 5.931904855s FAIL (2025-08-11 16:43:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.931925674s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................X...............................X....] duration: 2.013904185s FAIL (2025-08-11 16:43:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 2.013927438s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.866845234s FAIL (2025-08-11 16:43:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.86686479s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X............................................X.........] duration: 5.995135529s FAIL (2025-08-11 16:42:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.995162772s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.....X............................................X........] duration: 5.990327794s FAIL (2025-08-11 16:42:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.990353313s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X...............................X...] duration: 2.109502509s FAIL (2025-08-11 16:42:18 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 2.10952968s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.876907265s FAIL (2025-08-11 16:41:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.876929927s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.....X............................................X.......] duration: 6.047543567s FAIL (2025-08-11 16:41:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 6.047569095s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.....X............................................X......] duration: 5.621391303s FAIL (2025-08-11 16:41:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.62142076s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X...............................X..] duration: 1.893106733s FAIL (2025-08-11 16:41:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 1.893136219s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.818215215s FAIL (2025-08-11 16:40:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.818255673s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X...............................X.] duration: 1.886380391s FAIL (2025-08-11 16:40:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 1.886408355s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.....X............................................X.....] duration: 5.870814125s FAIL (2025-08-11 16:40:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.870841567s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.879928477s FAIL (2025-08-11 16:39:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.879960859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.....X............................................X....] duration: 6.053805007s FAIL (2025-08-11 16:39:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 6.053847278s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X...............................X] duration: 1.630434608s FAIL (2025-08-11 16:39:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 16:39:31.841821231 (71h13m26.272901362s) [upload=74240]: Starting upload Aug 11 16:39:31.841901154 (71h13m26.272981275s) [upload=74240]: Done starting upload Aug 11 16:39:31.841958824 (71h13m26.273038955s) [upload=74240]: Uploading segments... Aug 11 16:39:31.841998260 (71h13m26.273078381s) [upload=74240]: Got next segment. Inline: true Aug 11 16:39:31.842002949 (71h13m26.27308308s) [upload=74240]: Waiting for error group managing segments... Aug 11 16:39:31.842046892 (71h13m26.273127023s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:39:31.842052614 (71h13m26.273132745s) [upload=74240]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:39:31.842057693 (71h13m26.273137824s) [upload=74240]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:39:32.139444920 (71h13m26.570525051s) [upload=74240]: Done uploading segments... Aug 11 16:39:32.139467233 (71h13m26.570547374s) [upload=74240]: Upload finished. err: <nil> Duration:* 1.630460117s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.832117373s FAIL (2025-08-11 16:38:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.832150556s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.....X............................................X...] duration: 6.150883786s FAIL (2025-08-11 16:38:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 6.150906219s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.975112933s FAIL (2025-08-11 16:38:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 16:16:23.023938728 (70h50m17.455018849s) [upload=73843]: Starting upload Aug 11 16:16:23.024019263 (70h50m17.455099394s) [upload=73843]: Done starting upload Aug 11 16:16:23.024059239 (70h50m17.45513937s) [upload=73843]: Uploading segments... Aug 11 16:16:23.024092443 (70h50m17.455172574s) [upload=73843]: Got next segment. Inline: true Aug 11 16:16:23.024096220 (70h50m17.455176351s) [upload=73843]: Waiting for error group managing segments... Aug 11 16:16:23.024129163 (70h50m17.455209294s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:16:23.024133080 (70h50m17.455213211s) [upload=73843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:16:23.024135555 (70h50m17.455215686s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:16:23.508263261 (70h50m17.939343392s) [upload=73843]: Done uploading segments... Aug 11 16:16:23.508280414 (70h50m17.939360535s) [upload=73843]: Upload finished. err: <nil> Duration:* 1.975140925s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.907933567s FAIL (2025-08-11 16:37:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.907961522s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X.....X............................................X..] duration: 5.865604997s FAIL (2025-08-11 16:37:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.865629163s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.886727755s FAIL (2025-08-11 16:37:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 16:16:23.023938728 (70h50m17.455018849s) [upload=73843]: Starting upload Aug 11 16:16:23.024019263 (70h50m17.455099394s) [upload=73843]: Done starting upload Aug 11 16:16:23.024059239 (70h50m17.45513937s) [upload=73843]: Uploading segments... Aug 11 16:16:23.024092443 (70h50m17.455172574s) [upload=73843]: Got next segment. Inline: true Aug 11 16:16:23.024096220 (70h50m17.455176351s) [upload=73843]: Waiting for error group managing segments... Aug 11 16:16:23.024129163 (70h50m17.455209294s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:16:23.024133080 (70h50m17.455213211s) [upload=73843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:16:23.024135555 (70h50m17.455215686s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:16:23.508263261 (70h50m17.939343392s) [upload=73843]: Done uploading segments... Aug 11 16:16:23.508280414 (70h50m17.939360535s) [upload=73843]: Upload finished. err: <nil> Duration:* 1.886755547s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.900076531s FAIL (2025-08-11 16:36:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.90010146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 3.246213739s FAIL (2025-08-11 16:36:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 16:16:23.023938728 (70h50m17.455018849s) [upload=73843]: Starting upload Aug 11 16:16:23.024019263 (70h50m17.455099394s) [upload=73843]: Done starting upload Aug 11 16:16:23.024059239 (70h50m17.45513937s) [upload=73843]: Uploading segments... Aug 11 16:16:23.024092443 (70h50m17.455172574s) [upload=73843]: Got next segment. Inline: true Aug 11 16:16:23.024096220 (70h50m17.455176351s) [upload=73843]: Waiting for error group managing segments... Aug 11 16:16:23.024129163 (70h50m17.455209294s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:16:23.024133080 (70h50m17.455213211s) [upload=73843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:16:23.024135555 (70h50m17.455215686s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:16:23.508263261 (70h50m17.939343392s) [upload=73843]: Done uploading segments... Aug 11 16:16:23.508280414 (70h50m17.939360535s) [upload=73843]: Upload finished. err: <nil> Duration:* 3.246241082s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X.....X............................................X.] duration: 5.814248605s FAIL (2025-08-11 16:36:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 5.814276859s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.896188401s FAIL (2025-08-11 16:36:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.896211225s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.094273289s FAIL (2025-08-11 16:35:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 16:16:23.023938728 (70h50m17.455018849s) [upload=73843]: Starting upload Aug 11 16:16:23.024019263 (70h50m17.455099394s) [upload=73843]: Done starting upload Aug 11 16:16:23.024059239 (70h50m17.45513937s) [upload=73843]: Uploading segments... Aug 11 16:16:23.024092443 (70h50m17.455172574s) [upload=73843]: Got next segment. Inline: true Aug 11 16:16:23.024096220 (70h50m17.455176351s) [upload=73843]: Waiting for error group managing segments... Aug 11 16:16:23.024129163 (70h50m17.455209294s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:16:23.024133080 (70h50m17.455213211s) [upload=73843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:16:23.024135555 (70h50m17.455215686s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:16:23.508263261 (70h50m17.939343392s) [upload=73843]: Done uploading segments... Aug 11 16:16:23.508280414 (70h50m17.939360535s) [upload=73843]: Upload finished. err: <nil> Duration:* 2.094304179s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X.....X............................................X] duration: 4.241724997s FAIL (2025-08-11 16:35:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:35:23.944099565 (71h9m18.375179696s) [upload=74174]: Starting upload Aug 11 16:35:23.944142197 (71h9m18.375222328s) [upload=74174]: Done starting upload Aug 11 16:35:23.944290570 (71h9m18.375370701s) [upload=74174]: Uploading segments... Aug 11 16:35:23.944321590 (71h9m18.375401721s) [upload=74174]: Got next segment. Inline: false Aug 11 16:35:23.944344343 (71h9m18.375424474s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:35:23.944348621 (71h9m18.375428752s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:35:24.406399722 (71h9m18.837479854s) [seg_pos={0 0}, upload=74174]: Begin upload segment... Aug 11 16:35:24.406449387 (71h9m18.837529518s) [seg_pos={0 0}, upload=74174]: Done begin upload segment. Aug 11 16:35:24.406475717 (71h9m18.837555848s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406508711 (71h9m18.837588842s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.406863500 (71h9m18.837943631s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407026131 (71h9m18.838106252s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407142373 (71h9m18.838222494s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407420415 (71h9m18.838500536s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.407683579 (71h9m18.83876371s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408126587 (71h9m18.839206718s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408354082 (71h9m18.839434213s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408694463 (71h9m18.839774584s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.408921638 (71h9m18.840001769s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409132681 (71h9m18.840212812s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409381237 (71h9m18.840461368s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409593192 (71h9m18.840673323s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.409793966 (71h9m18.840874097s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410180757 (71h9m18.841260888s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410444992 (71h9m18.841525123s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:24.410673891 (71h9m18.841754022s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Uploading piece... Aug 11 16:35:25.618646236 (71h9m20.049726367s) [upload=74174]: Next returned nil segment Aug 11 16:35:25.618664400 (71h9m20.049744531s) [upload=74174]: Waiting for error group managing segments... Aug 11 16:35:25.927381945 (71h9m20.358462076s) [noise=true , address=193.149.164.143:4701 , piece_id=CCYW2PQ3QNUB2HNNFXXGS6EZ42OQ6TBUR4VRQ7R56TAS2CQJFLYQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.323160953 (71h9m20.754241084s) [noise=true , address=79.127.219.38:4026 , piece_id=WGDPBLYFFOCOXJQNSAAFALG5WC7YBDMTRYLYYYRD4P7KSR4VOD2Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377281556 (71h9m20.808361697s) [noise=true , address=104.36.82.61:4407 , piece_id=TFENYQT2DPY4JPFZLRFDOCCHR5FA6HSROYZD3VRB72Z63I7VT5JQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.377550461 (71h9m20.808630592s) [noise=true , address=104.36.82.35:4616 , piece_id=GEL377PWHMAUQHL7RB2WV56HXAYEK5WLG2KXPHUCRCU5OIMENGKA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.390996899 (71h9m20.82207702s) [noise=true , address=104.36.82.23:4446 , piece_id=YNSW7AD2TCFVV34TA2LPNXYJVZ6LMPT5HLD74CKWXWOKJJT6SEJQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.394608920 (71h9m20.825689051s) [noise=true , address=104.36.82.60:4367 , piece_id=4EJG3WIGQUYSGEYTZBA4D5I47WRBG2TSTLZYDBA4SKKT7I4SA4SA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.423529997 (71h9m20.854610128s) [noise=true , address=104.36.82.23:4445 , piece_id=4FCYSTHXOZDFLCARULPYFWOVENHISGBEIQOVUFEFOWLHVUPGNKTA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.436418768 (71h9m20.86749891s) [noise=true , address=104.36.82.8:4998 , piece_id=WVZVGKXVWPMYLJEF2HS7QHON5ZPNQCNN3ERZZOMGHTNGQHMEHR3Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.438386484 (71h9m20.869466615s) [noise=true , address=104.36.82.36:4644 , piece_id=KH5S33POXH2SVQRIPVD5ZMPE7BMZWHSFTOPNIGA7M4KHAGI6KHFQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.451348765 (71h9m20.882428896s) [noise=true , address=104.36.82.62:4415 , piece_id=TJ2BSMJ5USGM37TLTCTZAJKB5QDUWKPJR3RDRCBO5VD3EHIEAHFA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.487992629 (71h9m20.91907276s) [noise=true , address=104.36.82.46:4498 , piece_id=HDYMLA2Q3L4CE4C7XSKUBIWZJ5R2KXYCAJUFLMFRSM5QOYD74L6Q, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.495221812 (71h9m20.926301943s) [noise=true , address=104.36.82.39:4746 , piece_id=PBSDZCC7CTPOAH5OLYJFGT6TVCL4TNI3FZL5Z5MXGMQQ65L4GUBA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.506352639 (71h9m20.93743278s) [noise=true , address=104.36.82.34:4556 , piece_id=KZICWIYBDXN556SX5FUM6OJY3XUPVWJLEURW2CE2HNREGX7LDCGA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532639084 (71h9m20.963719215s) [noise=true , address=104.36.82.22:4510 , piece_id=3Z55FU2VBZ4IZAG524FGN3PBDYZSE6FWUBBK6ZLMJ2TCEL22H6RQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: <nil> Aug 11 16:35:26.532664734 (71h9m20.963744865s) [seg_pos={0 0}, upload=74174]: Segment reached optimal threshold of 14 pieces. Aug 11 16:35:26.532772590 (71h9m20.963852711s) [noise=true , address=104.36.82.44:4871 , piece_id=HZBEDUAOIYBUHMOZRDAUF4PEAAZJOA5WRSWALWA7UQIK3JDQSZ4A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532827094 (71h9m20.963907225s) [noise=true , address=77.90.31.13:4642 , piece_id=MQHLEOZWHY4TGHYNVP7F7IDAQ6F63SIIIWHSQKWQ6BN3JKP3AH5A, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532880677 (71h9m20.963960808s) [noise=true , address=77.90.31.14:4665 , piece_id=CCJO7T4LBP5WC6N77NLSL4HBUH5CFA6G6CSKYINRUE7NKDOP5EJA, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce): protocol: expected piece hash; context canceled; EOF Aug 11 16:35:26.532982181 (71h9m20.964062312s) [noise=true , address=104.36.82.38:4723 , piece_id=U5FJU52NE43DGAS5ZHCYWRLWURVDVNNDWA4JP4TCTKJGALGVFOGQ, seg_pos={0 0}, upload=74174]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SBKU1E8RjvAx3NzWEveNX5cfuQqq4SKgzY15eeJyfMV2DcF9i): protocol: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40914->104.36.82.38:4723: use of closed network connection Aug 11 16:35:26.533016367 (71h9m20.964096498s) [seg_pos={0 0}, upload=74174]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:35:26.533084618 (71h9m20.964164749s) [upload=74174]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:35:26.533088645 (71h9m20.964168776s) [upload=74174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:35:26.852780658 (71h9m21.283860789s) [upload=74174]: Done uploading segments... Aug 11 16:35:26.852799755 (71h9m21.283879886s) [upload=74174]: Upload finished. err: <nil> Duration: 4.241750697s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.815057206s FAIL (2025-08-11 16:35:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 11 16:20:42.438359750 (70h54m36.869439881s) [upload=73915]: Starting upload Aug 11 16:20:42.438414104 (70h54m36.869494225s) [upload=73915]: Done starting upload Aug 11 16:20:42.438489136 (70h54m36.869569267s) [upload=73915]: Uploading segments... Aug 11 16:20:42.438621770 (70h54m36.869701891s) [upload=73915]: Got next segment. Inline: false Aug 11 16:20:42.438664713 (70h54m36.869744844s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:20:42.438669001 (70h54m36.869749132s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:20:42.676790711 (70h54m37.107870842s) [seg_pos={0 0}, upload=73915]: Begin upload segment... Aug 11 16:20:42.676835908 (70h54m37.107916039s) [seg_pos={0 0}, upload=73915]: Done begin upload segment. Aug 11 16:20:42.676853832 (70h54m37.107933964s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676866747 (70h54m37.107946878s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676910421 (70h54m37.107990552s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.676958241 (70h54m37.108038383s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677329863 (70h54m37.108409994s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677368517 (70h54m37.108448648s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.677671015 (70h54m37.108751146s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678009293 (70h54m37.109089424s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678247930 (70h54m37.109328061s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678532164 (70h54m37.109612285s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678765730 (70h54m37.109845861s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.678993456 (70h54m37.110073587s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679139145 (70h54m37.110219266s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679376159 (70h54m37.11045629s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679589166 (70h54m37.110669297s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:42.679786334 (70h54m37.110866465s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Uploading piece... Aug 11 16:20:44.458058779 (70h54m38.88913891s) [upload=73915]: Next returned nil segment Aug 11 16:20:44.458076964 (70h54m38.889157095s) [upload=73915]: Waiting for error group managing segments... Aug 11 16:20:44.762791322 (70h54m39.193871453s) [noise=true , address=45.91.116.7:4010 , piece_id=I5FZKKDZEIJAEBGXVY5E4PSKY2XN6FFB6YJI7CTWGTS63FVWC77A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.819598246 (70h54m39.250678377s) [noise=true , address=45.91.116.4:4939 , piece_id=D3TLK6SUW4O7UUZVU6PADTAGQZ6SJHYAXYG2UULS7VZNMZ5YSOIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.877739022 (70h54m39.308819143s) [noise=true , address=77.90.31.11:4555 , piece_id=42V6RDCOSJRZYHIH7AUYX35F7GRXVZRHRUNBTI3ZDDQSVK2IYBAA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.936181786 (70h54m39.367261918s) [noise=true , address=45.91.116.10:4083 , piece_id=ACEAUOZZCKIA4LTESFID3ZWUKFVTJACUI6VB327RTMHJLTPZBYGQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.937836643 (70h54m39.368916774s) [noise=true , address=77.90.31.18:4807 , piece_id=JZTPSUJQ7H7AAQC2JQLY37UM2ZPSCFVYDNOG7RSCJ6SPTWCVVPLA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.939510475 (70h54m39.370590596s) [noise=true , address=77.90.31.17:4771 , piece_id=4Y664XUE5YWHPNAWUYSZ4BPISRV5YM5IGV4VK53RCPFZYEEBOKIQ, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.944277246 (70h54m39.375357377s) [noise=true , address=104.36.82.12:4151 , piece_id=QUY4X3H5VQPZXZ4JCHFKXLUETZXOBEN34KO7YZEMVYQEVGKR5ZZA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:44.945075824 (70h54m39.376155955s) [noise=true , address=45.91.116.13:4155 , piece_id=YTXBYJIN6WYX2LBRDYH7YU55DKC7ZLLECK3HI22IKBWFRT6ZHTIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.006188045 (70h54m39.437268176s) [noise=true , address=45.91.116.5:4298 , piece_id=QYRZXB5RPTYY3Y54CVYQA76YEOHCKIPOMOP36Z2W6PSF35JDULFA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.042863199 (70h54m39.47394333s) [noise=true , address=77.90.31.14:4663 , piece_id=I5W3UOQI6RQFDF2A5NGYG7GUXBP2XE3N6LCTBVHAXPEWBMVSV26Q, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.048316484 (70h54m39.479396645s) [noise=true , address=77.90.31.19:4843 , piece_id=NZ6GA4W7S4VGYLD5TBIOVCTFSBODMTOQY7SBY5OB2XWE766R2B3A, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.054276718 (70h54m39.485356839s) [noise=true , address=45.91.116.12:4227 , piece_id=C5NVQ4KJ4UVIIXGUUXYLYGZ6T3TVKESSRQX7KSYQZ7O5PYUS3XVA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056028469 (70h54m39.487108601s) [noise=true , address=45.91.116.13:4153 , piece_id=6FFUWDJYTJPUBXGHDGREUB4OM4DGX6COJTXHFWHLGW7QK3CO2JDA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: <nil> Aug 11 16:20:45.056044611 (70h54m39.487124732s) [seg_pos={0 0}, upload=73915]: Segment reached optimal threshold of 13 pieces. Aug 11 16:20:45.056165442 (70h54m39.487245573s) [noise=true , address=45.91.116.6:4974 , piece_id=GUI4A3KX4FASSTLQDMD4PHG75FQZRZEZCWNIMLWEJBBHQ5IH6SIA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056214636 (70h54m39.487294767s) [noise=true , address=104.36.82.10:4079 , piece_id=CYHDYCMVCYISFUDJOXGK2RCJVPRBZXXBAX3ICDJTH4OUPTQL6RMA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 11 16:20:45.056367308 (70h54m39.48744744s) [noise=true , address=104.36.82.5:4839 , piece_id=56H4YACEHZNRC4IGN6LQ2OHVYAL6DFVJ3E5XKXUMAWHO6UCMICGA, seg_pos={0 0}, upload=73915]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33142->104.36.82.5:4839: use of closed network connection Aug 11 16:20:45.056394020 (70h54m39.487474151s) [seg_pos={0 0}, upload=73915]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 11 16:20:45.056469904 (70h54m39.487550035s) [upload=73915]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:20:45.056474313 (70h54m39.487554444s) [upload=73915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:20:45.348914657 (70h54m39.779994788s) [upload=73915]: Done uploading segments... Aug 11 16:20:45.348932771 (70h54m39.780012902s) [upload=73915]: Upload finished. err: <nil> Duration: 5.815084487s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X.....X............................................] duration: 5.688369616s FAIL (2025-08-11 16:35:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 16:04:01.431038800 (70h37m55.862118931s) [upload=73622]: Starting upload Aug 11 16:04:01.431082304 (70h37m55.862162435s) [upload=73622]: Done starting upload Aug 11 16:04:01.431252088 (70h37m55.86233222s) [upload=73622]: Uploading segments... Aug 11 16:04:01.431284380 (70h37m55.862364511s) [upload=73622]: Got next segment. Inline: false Aug 11 16:04:01.431307364 (70h37m55.862387495s) [upload=73622]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:04:01.431312113 (70h37m55.862392254s) [upload=73622]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:04:01.890307323 (70h37m56.321387454s) [seg_pos={0 0}, upload=73622]: Begin upload segment... Aug 11 16:04:01.890353201 (70h37m56.321433332s) [seg_pos={0 0}, upload=73622]: Done begin upload segment. Aug 11 16:04:01.890375163 (70h37m56.321455294s) [noise=true , address=104.36.82.14:4227 , piece_id=EVYZRDJP4EXDXYOLOQANMJGNFIXOEJIU3VRZNI4AL6DHEYFKABUQ, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.890826506 (70h37m56.321906637s) [noise=true , address=104.36.82.15:4284 , piece_id=AUUFKETDOAVHZ55LJNOYXEYQZIGINANGNE6ZAG3VJGE4TCYJ4LTA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.891054562 (70h37m56.322134693s) [noise=true , address=104.36.82.3:4776 , piece_id=D7WWUBHGQJ6CBNVOUVBNUMRSI2M2GJMVBSBHKFCC2JGI3YC7ATKA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.891172759 (70h37m56.32225289s) [noise=true , address=104.36.82.46:4490 , piece_id=3NX3DMQX4LMIN6GGZUTLVBFIDKYVHV2JT523VXKJOKMZHRY3VFNQ, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.891361520 (70h37m56.322441651s) [noise=true , address=79.127.201.215:4351 , piece_id=I6W5OOHKCPZMH5LOBMAB3RBLKTNRZB63DBYSMBB6RMBIUXLO3ZQA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.891423829 (70h37m56.32250396s) [noise=true , address=104.36.82.62:4413 , piece_id=JIRMRDPS5NWAUATMLRDYMAPB3YVPZYNCE6342IFLGKOQCZM4SQVQ, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.891616818 (70h37m56.322696949s) [noise=true , address=104.36.82.23:4472 , piece_id=GO773SFRW3AMOFEY5HJA2KV5TV7AX7U62L67UWLWXTH4IECOQJ5A, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.891659941 (70h37m56.322740072s) [noise=true , address=79.127.205.234:4596 , piece_id=CU4MDPRZHFMHFKDWMXX35VKWBD3L4M65PMRNCWC2PW7UVUT6XWOA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.891882968 (70h37m56.322963099s) [noise=true , address=104.36.82.35:4591 , piece_id=I7DFGEU2JUZOSPYQ2OTPFDEYVQ3L6GBU7F5MZRKW6FBWJVMZ7YDA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.891939175 (70h37m56.323019306s) [noise=true , address=79.127.219.38:4036 , piece_id=LWTZ6NKTIFU6FZODTCGNSEQREXXU3JFHGKHYA6UYWS4NHETNJBKQ, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.892110543 (70h37m56.323190674s) [noise=true , address=104.36.82.4:4832 , piece_id=AWNCE6P4IZ5RHQV32ID5ZD56ALT3AXAOHONMBI7UTVIJICSYRFNA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.892242175 (70h37m56.323322306s) [noise=true , address=104.36.82.61:4389 , piece_id=HOJHE6BMJYDQKT33ME772774IPWOKXDUMA4EFBE6JPIROIDVGB2A, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.892536228 (70h37m56.323616379s) [noise=true , address=64.49.13.170:4720 , piece_id=VM36K6L5UGCHXOLW2VJZG7EDDYJJKBZ2CEDUBFZXES244YRQQPDA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.892809370 (70h37m56.323889511s) [noise=true , address=104.36.82.39:4738 , piece_id=I6QYXJM6JA7R3USVMSPGAQTHXBWHG5ELWXOOI7QG7K3Q7NRXCI6Q, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.893101749 (70h37m56.324181881s) [noise=true , address=79.127.219.51:4181 , piece_id=JWR2PV54ZET3SRXZ56U2U6SRZ2B2HHHZ6WMVJ5T7KULICMDO7BXA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.893388549 (70h37m56.32446878s) [noise=true , address=104.36.82.22:4543 , piece_id=ULDKBU3ZDCTHJCSERLUPSBDBB6QFRZZPVFNWKEZUWRNAMVNTJ7DA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.893619691 (70h37m56.324699822s) [noise=true , address=104.36.82.42:4812 , piece_id=P6L55YK4MWPEN33E3SI6HUUEFGVCOMFC3IFVO3YZVBXT5TCTE3WQ, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:01.893923442 (70h37m56.325003573s) [noise=true , address=104.36.82.59:4320 , piece_id=ZXA7ZBTH55YHBZ6Q4SQCKFBYPCLCITMYDWHY7TL2TSRAU67RH5YA, seg_pos={0 0}, upload=73622]: Uploading piece... Aug 11 16:04:03.014075253 (70h37m57.445155404s) [upload=73622]: Next returned nil segment Aug 11 16:04:03.014097896 (70h37m57.445178057s) [upload=73622]: Waiting for error group managing segments... Aug 11 16:04:03.297786654 (70h37m57.728866795s) [noise=true , address=79.127.205.234:4596 , piece_id=CU4MDPRZHFMHFKDWMXX35VKWBD3L4M65PMRNCWC2PW7UVUT6XWOA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.402824193 (70h37m57.833904324s) [noise=true , address=64.49.13.170:4720 , piece_id=VM36K6L5UGCHXOLW2VJZG7EDDYJJKBZ2CEDUBFZXES244YRQQPDA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.777113195 (70h37m58.208193326s) [noise=true , address=79.127.219.51:4181 , piece_id=JWR2PV54ZET3SRXZ56U2U6SRZ2B2HHHZ6WMVJ5T7KULICMDO7BXA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.830373563 (70h37m58.261453694s) [noise=true , address=79.127.219.38:4036 , piece_id=LWTZ6NKTIFU6FZODTCGNSEQREXXU3JFHGKHYA6UYWS4NHETNJBKQ, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.851036259 (70h37m58.28211639s) [noise=true , address=104.36.82.42:4812 , piece_id=P6L55YK4MWPEN33E3SI6HUUEFGVCOMFC3IFVO3YZVBXT5TCTE3WQ, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.863968733 (70h37m58.295048854s) [noise=true , address=104.36.82.3:4776 , piece_id=D7WWUBHGQJ6CBNVOUVBNUMRSI2M2GJMVBSBHKFCC2JGI3YC7ATKA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.867913050 (70h37m58.298993191s) [noise=true , address=104.36.82.59:4320 , piece_id=ZXA7ZBTH55YHBZ6Q4SQCKFBYPCLCITMYDWHY7TL2TSRAU67RH5YA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.870715814 (70h37m58.301795975s) [noise=true , address=79.127.201.215:4351 , piece_id=I6W5OOHKCPZMH5LOBMAB3RBLKTNRZB63DBYSMBB6RMBIUXLO3ZQA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.877046257 (70h37m58.308126388s) [noise=true , address=104.36.82.23:4472 , piece_id=GO773SFRW3AMOFEY5HJA2KV5TV7AX7U62L67UWLWXTH4IECOQJ5A, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.877830407 (70h37m58.308910548s) [noise=true , address=104.36.82.4:4832 , piece_id=AWNCE6P4IZ5RHQV32ID5ZD56ALT3AXAOHONMBI7UTVIJICSYRFNA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.909950384 (70h37m58.341030516s) [noise=true , address=104.36.82.15:4284 , piece_id=AUUFKETDOAVHZ55LJNOYXEYQZIGINANGNE6ZAG3VJGE4TCYJ4LTA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.935330317 (70h37m58.366410438s) [noise=true , address=104.36.82.39:4738 , piece_id=I6QYXJM6JA7R3USVMSPGAQTHXBWHG5ELWXOOI7QG7K3Q7NRXCI6Q, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.937536838 (70h37m58.368616969s) [noise=true , address=104.36.82.62:4413 , piece_id=JIRMRDPS5NWAUATMLRDYMAPB3YVPZYNCE6342IFLGKOQCZM4SQVQ, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.941025543 (70h37m58.372105664s) [noise=true , address=104.36.82.14:4227 , piece_id=EVYZRDJP4EXDXYOLOQANMJGNFIXOEJIU3VRZNI4AL6DHEYFKABUQ, seg_pos={0 0}, upload=73622]: Done uploading piece. err: <nil> Aug 11 16:04:03.941037427 (70h37m58.372117558s) [seg_pos={0 0}, upload=73622]: Segment reached optimal threshold of 14 pieces. Aug 11 16:04:03.941205719 (70h37m58.37228585s) [noise=true , address=104.36.82.46:4490 , piece_id=3NX3DMQX4LMIN6GGZUTLVBFIDKYVHV2JT523VXKJOKMZHRY3VFNQ, seg_pos={0 0}, upload=73622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PUxwz3cZkF8e2VkjTXnu1utrFzQRTxdohfd9EvDaLvyejpTji): protocol: expected piece hash; context canceled; EOF Aug 11 16:04:03.941278699 (70h37m58.37235883s) [noise=true , address=104.36.82.22:4543 , piece_id=ULDKBU3ZDCTHJCSERLUPSBDBB6QFRZZPVFNWKEZUWRNAMVNTJ7DA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19iUYyo8cGDLGDnvdNo2Qfj1K6tp6EJWF5U6rXD4gJ7Hv7Gsch): protocol: expected piece hash; context canceled; EOF Aug 11 16:04:03.941306892 (70h37m58.372387024s) [noise=true , address=104.36.82.35:4591 , piece_id=I7DFGEU2JUZOSPYQ2OTPFDEYVQ3L6GBU7F5MZRKW6FBWJVMZ7YDA, seg_pos={0 0}, upload=73622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a453d6xSb26epbgqPuco3UPTf9yzoxBeKToCUMNiNHUddAq55): protocol: expected piece hash; context canceled; EOF Aug 11 16:04:03.941346147 (70h37m58.372426278s) [noise=true , address=104.36.82.61:4389 , piece_id=HOJHE6BMJYDQKT33ME772774IPWOKXDUMA4EFBE6JPIROIDVGB2A, seg_pos={0 0}, upload=73622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VbbhJD2Dga9GactoTgUPgDaKMq9p7YLKxjPtRhgkVbH4DKzpZ): protocol: expected piece hash; context canceled; EOF Aug 11 16:04:03.941378028 (70h37m58.372458159s) [seg_pos={0 0}, upload=73622]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:04:03.941439325 (70h37m58.372519456s) [upload=73622]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:04:03.941443564 (70h37m58.372523695s) [upload=73622]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:04:04.967670377 (70h37m59.398750508s) [upload=73622]: Done uploading segments... Aug 11 16:04:04.967688492 (70h37m59.398768613s) [upload=73622]: Upload finished. err: <nil> Duration: 5.688401718s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.968803677s FAIL (2025-08-11 16:35:00 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 11 16:16:23.023938728 (70h50m17.455018849s) [upload=73843]: Starting upload Aug 11 16:16:23.024019263 (70h50m17.455099394s) [upload=73843]: Done starting upload Aug 11 16:16:23.024059239 (70h50m17.45513937s) [upload=73843]: Uploading segments... Aug 11 16:16:23.024092443 (70h50m17.455172574s) [upload=73843]: Got next segment. Inline: true Aug 11 16:16:23.024096220 (70h50m17.455176351s) [upload=73843]: Waiting for error group managing segments... Aug 11 16:16:23.024129163 (70h50m17.455209294s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:16:23.024133080 (70h50m17.455213211s) [upload=73843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 16:16:23.024135555 (70h50m17.455215686s) [upload=73843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:16:23.508263261 (70h50m17.939343392s) [upload=73843]: Done uploading segments... Aug 11 16:16:23.508280414 (70h50m17.939360535s) [upload=73843]: Upload finished. err: <nil> Duration:* 1.968837502s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)