Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.498437619s FAIL (2025-08-11 14:32:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.498459773s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............X..............................X.X...X..X.....X] duration: 5.049346535s FAIL (2025-08-11 14:31:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 14:31:28.797649884 (69h5m23.228730005s) [upload=72008]: Starting upload Aug 11 14:31:28.797724437 (69h5m23.228804568s) [upload=72008]: Done starting upload Aug 11 14:31:28.797985085 (69h5m23.229065216s) [upload=72008]: Uploading segments... Aug 11 14:31:28.798029571 (69h5m23.229109702s) [upload=72008]: Got next segment. Inline: false Aug 11 14:31:28.798057073 (69h5m23.229137204s) [upload=72008]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:31:28.798061191 (69h5m23.229141322s) [upload=72008]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:31:29.295888418 (69h5m23.726968549s) [seg_pos={0 0}, upload=72008]: Begin upload segment... Aug 11 14:31:29.295938112 (69h5m23.727018243s) [seg_pos={0 0}, upload=72008]: Done begin upload segment. Aug 11 14:31:29.295959162 (69h5m23.727039293s) [noise=true , address=45.91.116.7:4039 , piece_id=TG65XLLASGXWBNODCL5MCOODUZXTZ23TOHD6ZM7XFZ6K3CJWIHEQ, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.296001143 (69h5m23.727081274s) [noise=true , address=104.36.82.39:4730 , piece_id=72CEQIFEDHHDEPM2IYWUYZCZRZZLTJ27TGSBE6J5MFVZZ47CUF7Q, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.296383876 (69h5m23.727464007s) [noise=true , address=104.36.82.59:4302 , piece_id=S6DC2ATWBPANXV5TKHY65YNDXQJHUFKFNWDDWNVRIP5ZNVDUCJHQ, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.296482904 (69h5m23.727563035s) [noise=true , address=104.36.82.17:4332 , piece_id=CWZWQNAJ7MQUEZU2IGMBT4XWGKBXX3J3EMJVXE5R34PXDOSS6UGA, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.296676134 (69h5m23.727756255s) [noise=true , address=104.36.82.46:4508 , piece_id=7MV3QXE43MYOZJBIIDQVFVHWNOU7O32X76AX5RIO5DL2O67AGNWA, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.296800813 (69h5m23.727880944s) [noise=true , address=104.36.82.60:4352 , piece_id=4J6WDOP6LV6KI3HXMT4GD6TOJBIXFCMQS5OMACHSFXWTAGUOJMUQ, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.296970838 (69h5m23.728050969s) [noise=true , address=104.36.82.47:4528 , piece_id=57YDF5V7ATJVUSPTQSVGDRQUXTRQRG6EZ2IXZ2ILLMONF23OHL2Q, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.297186100 (69h5m23.728266231s) [noise=true , address=104.36.82.61:4382 , piece_id=EDCZDJ2C7IGLCMY3H6IBAP5E2RML7KRWOTVMXPVPFCJFSE3AFUPQ, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.297256535 (69h5m23.728336667s) [noise=true , address=104.36.82.5:4868 , piece_id=6Z2FF4CKEKHMANLDVQ4T5WDKCZJRAVJ2T3IAGWJM4HAVTSXITLRQ, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.297490783 (69h5m23.728570914s) [noise=true , address=104.36.82.42:4799 , piece_id=E7KYT6ZOIDZOABOMQ4RGF6OT66NHDNDI6FSM42XOMJ2NHYSIBBGA, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.297550768 (69h5m23.728630899s) [noise=true , address=104.36.82.62:4424 , piece_id=L3ZLBFDM6IHAA3UI3EP3V5SRKLRW5O7L5RVF4OYWQSMTB3MAWEEQ, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.297822077 (69h5m23.728902208s) [noise=true , address=104.36.82.19:4401 , piece_id=7DEYSAK2NJAZUV4CS4E7MMFPZRD6W4M6S7LAARVU4SH66MYQNRUA, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.298064261 (69h5m23.729144392s) [noise=true , address=104.36.82.2:4744 , piece_id=RCFYTR2CGUOOEMSBQ5T54DDSA2CI7UAGGSRPK3SIJFI64TT7JCDA, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.298290163 (69h5m23.729370284s) [noise=true , address=104.36.82.44:4873 , piece_id=5IRE2BYR3S7P3SLQWLJTGT6464P4PPDJUERD3U2ZD7FJ6WCZDQEQ, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.298513089 (69h5m23.72959321s) [noise=true , address=104.36.82.18:4379 , piece_id=IGMBV3UKBYLQJF4AQ4Q5EKNVWSSAXKAMCDAD6UK54SQBL4XR42DA, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.298712121 (69h5m23.729792252s) [noise=true , address=104.36.82.23:4462 , piece_id=PIVYRW7X7TX2B6UPJBI2ZI72EYK337D526RAX6LFGQXJORMCLCBA, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.298911071 (69h5m23.729991202s) [noise=true , address=104.36.82.37:4670 , piece_id=RAUSLUG6OG5GMMRH3RHL6EO7CYV7EPBXNWSCK3XTIMMBHYTVDO2A, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:29.299112526 (69h5m23.730192657s) [noise=true , address=104.36.82.34:4558 , piece_id=64VYAMVYN3ZCBBDW5TVJLFVPNHOKMJAZTPBFWULVONTNYNOSB7WQ, seg_pos={0 0}, upload=72008]: Uploading piece... Aug 11 14:31:30.883303543 (69h5m25.314383674s) [upload=72008]: Next returned nil segment Aug 11 14:31:30.883319604 (69h5m25.314399725s) [upload=72008]: Waiting for error group managing segments... Aug 11 14:31:31.318555088 (69h5m25.749635219s) [noise=true , address=45.91.116.7:4039 , piece_id=TG65XLLASGXWBNODCL5MCOODUZXTZ23TOHD6ZM7XFZ6K3CJWIHEQ, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.368005896 (69h5m25.799086027s) [noise=true , address=104.36.82.19:4401 , piece_id=7DEYSAK2NJAZUV4CS4E7MMFPZRD6W4M6S7LAARVU4SH66MYQNRUA, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.394334471 (69h5m25.825414593s) [noise=true , address=104.36.82.18:4379 , piece_id=IGMBV3UKBYLQJF4AQ4Q5EKNVWSSAXKAMCDAD6UK54SQBL4XR42DA, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.418160058 (69h5m25.84924018s) [noise=true , address=104.36.82.61:4382 , piece_id=EDCZDJ2C7IGLCMY3H6IBAP5E2RML7KRWOTVMXPVPFCJFSE3AFUPQ, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.426574378 (69h5m25.857654509s) [noise=true , address=104.36.82.34:4558 , piece_id=64VYAMVYN3ZCBBDW5TVJLFVPNHOKMJAZTPBFWULVONTNYNOSB7WQ, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.482363172 (69h5m25.913443303s) [noise=true , address=104.36.82.47:4528 , piece_id=57YDF5V7ATJVUSPTQSVGDRQUXTRQRG6EZ2IXZ2ILLMONF23OHL2Q, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.614731051 (69h5m26.045811182s) [noise=true , address=104.36.82.46:4508 , piece_id=7MV3QXE43MYOZJBIIDQVFVHWNOU7O32X76AX5RIO5DL2O67AGNWA, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.665634257 (69h5m26.096714388s) [noise=true , address=104.36.82.42:4799 , piece_id=E7KYT6ZOIDZOABOMQ4RGF6OT66NHDNDI6FSM42XOMJ2NHYSIBBGA, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.775031383 (69h5m26.206111514s) [noise=true , address=104.36.82.23:4462 , piece_id=PIVYRW7X7TX2B6UPJBI2ZI72EYK337D526RAX6LFGQXJORMCLCBA, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.800464647 (69h5m26.231544778s) [noise=true , address=104.36.82.17:4332 , piece_id=CWZWQNAJ7MQUEZU2IGMBT4XWGKBXX3J3EMJVXE5R34PXDOSS6UGA, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.807272012 (69h5m26.238352143s) [noise=true , address=104.36.82.37:4670 , piece_id=RAUSLUG6OG5GMMRH3RHL6EO7CYV7EPBXNWSCK3XTIMMBHYTVDO2A, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:31.861123109 (69h5m26.29220323s) [noise=true , address=104.36.82.59:4302 , piece_id=S6DC2ATWBPANXV5TKHY65YNDXQJHUFKFNWDDWNVRIP5ZNVDUCJHQ, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:32.159421814 (69h5m26.590501945s) [noise=true , address=104.36.82.62:4424 , piece_id=L3ZLBFDM6IHAA3UI3EP3V5SRKLRW5O7L5RVF4OYWQSMTB3MAWEEQ, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:32.217310826 (69h5m26.648390958s) [noise=true , address=104.36.82.39:4730 , piece_id=72CEQIFEDHHDEPM2IYWUYZCZRZZLTJ27TGSBE6J5MFVZZ47CUF7Q, seg_pos={0 0}, upload=72008]: Done uploading piece. err: <nil> Aug 11 14:31:32.217332729 (69h5m26.64841285s) [seg_pos={0 0}, upload=72008]: Segment reached optimal threshold of 14 pieces. Aug 11 14:31:32.217521760 (69h5m26.648601891s) [noise=true , address=104.36.82.2:4744 , piece_id=RCFYTR2CGUOOEMSBQ5T54DDSA2CI7UAGGSRPK3SIJFI64TT7JCDA, seg_pos={0 0}, upload=72008]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A6abQMwnX4PhLYTvkGLinPx4eNeVyS15SGzWiLUq4cwKQ6bmH): protocol: expected piece hash; context canceled; EOF Aug 11 14:31:32.217656157 (69h5m26.648736288s) [noise=true , address=104.36.82.60:4352 , piece_id=4J6WDOP6LV6KI3HXMT4GD6TOJBIXFCMQS5OMACHSFXWTAGUOJMUQ, seg_pos={0 0}, upload=72008]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Aug 11 14:31:32.217677418 (69h5m26.648757549s) [noise=true , address=104.36.82.44:4873 , piece_id=5IRE2BYR3S7P3SLQWLJTGT6464P4PPDJUERD3U2ZD7FJ6WCZDQEQ, seg_pos={0 0}, upload=72008]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo): protocol: expected piece hash; context canceled; EOF Aug 11 14:31:32.217763021 (69h5m26.648843152s) [noise=true , address=104.36.82.5:4868 , piece_id=6Z2FF4CKEKHMANLDVQ4T5WDKCZJRAVJ2T3IAGWJM4HAVTSXITLRQ, seg_pos={0 0}, upload=72008]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: write tcp 188.166.225.170:45178->104.36.82.5:4868: use of closed network connection; write tcp 188.166.225.170:45178->104.36.82.5:4868: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45178->104.36.82.5:4868: use of closed network connection Aug 11 14:31:32.217773782 (69h5m26.648853913s) [seg_pos={0 0}, upload=72008]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:31:32.217887830 (69h5m26.648967962s) [upload=72008]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:31:32.217908701 (69h5m26.648988832s) [upload=72008]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:31:32.734435244 (69h5m27.165515396s) [upload=72008]: Done uploading segments... Aug 11 14:31:32.734459310 (69h5m27.165539441s) [upload=72008]: Upload finished. err: <nil> Duration: 5.049383657s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........X......X................X............X...........] duration: 2.410183221s FAIL (2025-08-11 14:31:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 2.410215533s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 11.615768026s FAIL (2025-08-11 14:31:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.615787994s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............X..............................X.X...X..X.....] duration: 7.676449973s FAIL (2025-08-11 14:30:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after a8a9a4147ef29bc30a9cc6f1443217d7c90a66fe5704ef01df22a945c16fc6b8 != before acdb39bd3ed6066fc55ed4307594725cc9b228bd84f2243570a7e685d7fa13c9 ======LOGS====== Aug 11 14:26:34.154636911 (69h0m28.585717032s) [upload=71926]: Starting upload Aug 11 14:26:34.154683761 (69h0m28.585763892s) [upload=71926]: Done starting upload Aug 11 14:26:34.154779383 (69h0m28.585859515s) [upload=71926]: Uploading segments... Aug 11 14:26:34.154897640 (69h0m28.585977781s) [upload=71926]: Got next segment. Inline: false Aug 11 14:26:34.154948438 (69h0m28.586028569s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:26:34.154952836 (69h0m28.586032967s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:26:34.622954003 (69h0m29.054034124s) [seg_pos={0 0}, upload=71926]: Begin upload segment... Aug 11 14:26:34.622999981 (69h0m29.054080102s) [seg_pos={0 0}, upload=71926]: Done begin upload segment. Aug 11 14:26:34.623018938 (69h0m29.054099069s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623032263 (69h0m29.054112394s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623374248 (69h0m29.054454379s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623638803 (69h0m29.054718934s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623889944 (69h0m29.054970075s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624133850 (69h0m29.055213982s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624515461 (69h0m29.055595592s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624884006 (69h0m29.055964138s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625199660 (69h0m29.056279791s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625528379 (69h0m29.0566085s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625730746 (69h0m29.056810877s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626031792 (69h0m29.057111923s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626240461 (69h0m29.057320593s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626436196 (69h0m29.057516327s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626633714 (69h0m29.057713855s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626823427 (69h0m29.057903558s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627004864 (69h0m29.058084995s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627181031 (69h0m29.058261152s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:35.769773497 (69h0m30.200853628s) [upload=71926]: Next returned nil segment Aug 11 14:26:35.769791913 (69h0m30.200872044s) [upload=71926]: Waiting for error group managing segments... Aug 11 14:26:36.064698127 (69h0m30.495778258s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.134838792 (69h0m30.565918923s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.159959366 (69h0m30.591039497s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.568821428 (69h0m30.999901559s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.599190225 (69h0m31.030270356s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.611299104 (69h0m31.042379225s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.647379849 (69h0m31.07845997s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.665596635 (69h0m31.096676766s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.674096939 (69h0m31.10517707s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.702750032 (69h0m31.133830163s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.709991498 (69h0m31.141071629s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.764409790 (69h0m31.195489921s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.768074693 (69h0m31.199154824s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791039483 (69h0m31.222119614s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791060033 (69h0m31.222140154s) [seg_pos={0 0}, upload=71926]: Segment reached optimal threshold of 14 pieces. Aug 11 14:26:36.791177988 (69h0m31.222258119s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791238053 (69h0m31.222318184s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BwQctDb8mxcmcvf6jvTdQqPx7xv4QMsbaVimioZxGS5vstuvk): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791276257 (69h0m31.222356388s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791374634 (69h0m31.222454765s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aTYNxPbXJGM29Xqovc5VK4cun64wr2DoduaVCvKDj38xCLGKJ): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791427756 (69h0m31.222507887s) [seg_pos={0 0}, upload=71926]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:26:36.791519662 (69h0m31.222599793s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:26:36.791524552 (69h0m31.222604683s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:26:37.109408981 (69h0m31.540489112s) [upload=71926]: Done uploading segments... Aug 11 14:26:37.109426455 (69h0m31.540506586s) [upload=71926]: Upload finished. err: <nil> Duration: 7.676477095s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............X......X................X............X..........] duration: 2.308020507s FAIL (2025-08-11 14:30:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 2.30805428s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............X..............................X.X...X..X....] duration: 7.282352882s FAIL (2025-08-11 14:30:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after a8a9a4147ef29bc30a9cc6f1443217d7c90a66fe5704ef01df22a945c16fc6b8 != before acdb39bd3ed6066fc55ed4307594725cc9b228bd84f2243570a7e685d7fa13c9 ======LOGS====== Aug 11 14:26:34.154636911 (69h0m28.585717032s) [upload=71926]: Starting upload Aug 11 14:26:34.154683761 (69h0m28.585763892s) [upload=71926]: Done starting upload Aug 11 14:26:34.154779383 (69h0m28.585859515s) [upload=71926]: Uploading segments... Aug 11 14:26:34.154897640 (69h0m28.585977781s) [upload=71926]: Got next segment. Inline: false Aug 11 14:26:34.154948438 (69h0m28.586028569s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:26:34.154952836 (69h0m28.586032967s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:26:34.622954003 (69h0m29.054034124s) [seg_pos={0 0}, upload=71926]: Begin upload segment... Aug 11 14:26:34.622999981 (69h0m29.054080102s) [seg_pos={0 0}, upload=71926]: Done begin upload segment. Aug 11 14:26:34.623018938 (69h0m29.054099069s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623032263 (69h0m29.054112394s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623374248 (69h0m29.054454379s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623638803 (69h0m29.054718934s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623889944 (69h0m29.054970075s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624133850 (69h0m29.055213982s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624515461 (69h0m29.055595592s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624884006 (69h0m29.055964138s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625199660 (69h0m29.056279791s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625528379 (69h0m29.0566085s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625730746 (69h0m29.056810877s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626031792 (69h0m29.057111923s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626240461 (69h0m29.057320593s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626436196 (69h0m29.057516327s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626633714 (69h0m29.057713855s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626823427 (69h0m29.057903558s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627004864 (69h0m29.058084995s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627181031 (69h0m29.058261152s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:35.769773497 (69h0m30.200853628s) [upload=71926]: Next returned nil segment Aug 11 14:26:35.769791913 (69h0m30.200872044s) [upload=71926]: Waiting for error group managing segments... Aug 11 14:26:36.064698127 (69h0m30.495778258s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.134838792 (69h0m30.565918923s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.159959366 (69h0m30.591039497s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.568821428 (69h0m30.999901559s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.599190225 (69h0m31.030270356s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.611299104 (69h0m31.042379225s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.647379849 (69h0m31.07845997s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.665596635 (69h0m31.096676766s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.674096939 (69h0m31.10517707s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.702750032 (69h0m31.133830163s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.709991498 (69h0m31.141071629s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.764409790 (69h0m31.195489921s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.768074693 (69h0m31.199154824s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791039483 (69h0m31.222119614s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791060033 (69h0m31.222140154s) [seg_pos={0 0}, upload=71926]: Segment reached optimal threshold of 14 pieces. Aug 11 14:26:36.791177988 (69h0m31.222258119s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791238053 (69h0m31.222318184s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BwQctDb8mxcmcvf6jvTdQqPx7xv4QMsbaVimioZxGS5vstuvk): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791276257 (69h0m31.222356388s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791374634 (69h0m31.222454765s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aTYNxPbXJGM29Xqovc5VK4cun64wr2DoduaVCvKDj38xCLGKJ): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791427756 (69h0m31.222507887s) [seg_pos={0 0}, upload=71926]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:26:36.791519662 (69h0m31.222599793s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:26:36.791524552 (69h0m31.222604683s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:26:37.109408981 (69h0m31.540489112s) [upload=71926]: Done uploading segments... Aug 11 14:26:37.109426455 (69h0m31.540506586s) [upload=71926]: Upload finished. err: <nil> Duration: 7.282397708s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............X......X................X............X.........] duration: 1.885439007s FAIL (2025-08-11 14:30:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 1.885463996s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.357504792s FAIL (2025-08-11 14:30:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.357522768s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.970240737s FAIL (2025-08-11 14:29:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.970265254s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..............X......X................X............X........] duration: 1.916866486s FAIL (2025-08-11 14:29:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 1.916896763s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [................X..............................X.X...X..X...] duration: 6.742498844s FAIL (2025-08-11 14:29:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after a8a9a4147ef29bc30a9cc6f1443217d7c90a66fe5704ef01df22a945c16fc6b8 != before acdb39bd3ed6066fc55ed4307594725cc9b228bd84f2243570a7e685d7fa13c9 ======LOGS====== Aug 11 14:26:34.154636911 (69h0m28.585717032s) [upload=71926]: Starting upload Aug 11 14:26:34.154683761 (69h0m28.585763892s) [upload=71926]: Done starting upload Aug 11 14:26:34.154779383 (69h0m28.585859515s) [upload=71926]: Uploading segments... Aug 11 14:26:34.154897640 (69h0m28.585977781s) [upload=71926]: Got next segment. Inline: false Aug 11 14:26:34.154948438 (69h0m28.586028569s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:26:34.154952836 (69h0m28.586032967s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:26:34.622954003 (69h0m29.054034124s) [seg_pos={0 0}, upload=71926]: Begin upload segment... Aug 11 14:26:34.622999981 (69h0m29.054080102s) [seg_pos={0 0}, upload=71926]: Done begin upload segment. Aug 11 14:26:34.623018938 (69h0m29.054099069s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623032263 (69h0m29.054112394s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623374248 (69h0m29.054454379s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623638803 (69h0m29.054718934s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623889944 (69h0m29.054970075s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624133850 (69h0m29.055213982s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624515461 (69h0m29.055595592s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624884006 (69h0m29.055964138s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625199660 (69h0m29.056279791s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625528379 (69h0m29.0566085s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625730746 (69h0m29.056810877s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626031792 (69h0m29.057111923s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626240461 (69h0m29.057320593s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626436196 (69h0m29.057516327s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626633714 (69h0m29.057713855s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626823427 (69h0m29.057903558s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627004864 (69h0m29.058084995s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627181031 (69h0m29.058261152s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:35.769773497 (69h0m30.200853628s) [upload=71926]: Next returned nil segment Aug 11 14:26:35.769791913 (69h0m30.200872044s) [upload=71926]: Waiting for error group managing segments... Aug 11 14:26:36.064698127 (69h0m30.495778258s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.134838792 (69h0m30.565918923s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.159959366 (69h0m30.591039497s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.568821428 (69h0m30.999901559s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.599190225 (69h0m31.030270356s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.611299104 (69h0m31.042379225s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.647379849 (69h0m31.07845997s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.665596635 (69h0m31.096676766s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.674096939 (69h0m31.10517707s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.702750032 (69h0m31.133830163s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.709991498 (69h0m31.141071629s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.764409790 (69h0m31.195489921s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.768074693 (69h0m31.199154824s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791039483 (69h0m31.222119614s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791060033 (69h0m31.222140154s) [seg_pos={0 0}, upload=71926]: Segment reached optimal threshold of 14 pieces. Aug 11 14:26:36.791177988 (69h0m31.222258119s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791238053 (69h0m31.222318184s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BwQctDb8mxcmcvf6jvTdQqPx7xv4QMsbaVimioZxGS5vstuvk): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791276257 (69h0m31.222356388s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791374634 (69h0m31.222454765s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aTYNxPbXJGM29Xqovc5VK4cun64wr2DoduaVCvKDj38xCLGKJ): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791427756 (69h0m31.222507887s) [seg_pos={0 0}, upload=71926]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:26:36.791519662 (69h0m31.222599793s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:26:36.791524552 (69h0m31.222604683s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:26:37.109408981 (69h0m31.540489112s) [upload=71926]: Done uploading segments... Aug 11 14:26:37.109426455 (69h0m31.540506586s) [upload=71926]: Upload finished. err: <nil> Duration: 6.742523701s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................X..............................X.X...X..X..] duration: 7.018782952s FAIL (2025-08-11 14:28:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after a8a9a4147ef29bc30a9cc6f1443217d7c90a66fe5704ef01df22a945c16fc6b8 != before acdb39bd3ed6066fc55ed4307594725cc9b228bd84f2243570a7e685d7fa13c9 ======LOGS====== Aug 11 14:26:34.154636911 (69h0m28.585717032s) [upload=71926]: Starting upload Aug 11 14:26:34.154683761 (69h0m28.585763892s) [upload=71926]: Done starting upload Aug 11 14:26:34.154779383 (69h0m28.585859515s) [upload=71926]: Uploading segments... Aug 11 14:26:34.154897640 (69h0m28.585977781s) [upload=71926]: Got next segment. Inline: false Aug 11 14:26:34.154948438 (69h0m28.586028569s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:26:34.154952836 (69h0m28.586032967s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:26:34.622954003 (69h0m29.054034124s) [seg_pos={0 0}, upload=71926]: Begin upload segment... Aug 11 14:26:34.622999981 (69h0m29.054080102s) [seg_pos={0 0}, upload=71926]: Done begin upload segment. Aug 11 14:26:34.623018938 (69h0m29.054099069s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623032263 (69h0m29.054112394s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623374248 (69h0m29.054454379s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623638803 (69h0m29.054718934s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623889944 (69h0m29.054970075s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624133850 (69h0m29.055213982s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624515461 (69h0m29.055595592s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624884006 (69h0m29.055964138s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625199660 (69h0m29.056279791s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625528379 (69h0m29.0566085s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625730746 (69h0m29.056810877s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626031792 (69h0m29.057111923s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626240461 (69h0m29.057320593s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626436196 (69h0m29.057516327s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626633714 (69h0m29.057713855s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626823427 (69h0m29.057903558s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627004864 (69h0m29.058084995s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627181031 (69h0m29.058261152s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:35.769773497 (69h0m30.200853628s) [upload=71926]: Next returned nil segment Aug 11 14:26:35.769791913 (69h0m30.200872044s) [upload=71926]: Waiting for error group managing segments... Aug 11 14:26:36.064698127 (69h0m30.495778258s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.134838792 (69h0m30.565918923s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.159959366 (69h0m30.591039497s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.568821428 (69h0m30.999901559s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.599190225 (69h0m31.030270356s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.611299104 (69h0m31.042379225s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.647379849 (69h0m31.07845997s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.665596635 (69h0m31.096676766s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.674096939 (69h0m31.10517707s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.702750032 (69h0m31.133830163s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.709991498 (69h0m31.141071629s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.764409790 (69h0m31.195489921s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.768074693 (69h0m31.199154824s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791039483 (69h0m31.222119614s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791060033 (69h0m31.222140154s) [seg_pos={0 0}, upload=71926]: Segment reached optimal threshold of 14 pieces. Aug 11 14:26:36.791177988 (69h0m31.222258119s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791238053 (69h0m31.222318184s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BwQctDb8mxcmcvf6jvTdQqPx7xv4QMsbaVimioZxGS5vstuvk): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791276257 (69h0m31.222356388s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791374634 (69h0m31.222454765s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aTYNxPbXJGM29Xqovc5VK4cun64wr2DoduaVCvKDj38xCLGKJ): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791427756 (69h0m31.222507887s) [seg_pos={0 0}, upload=71926]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:26:36.791519662 (69h0m31.222599793s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:26:36.791524552 (69h0m31.222604683s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:26:37.109408981 (69h0m31.540489112s) [upload=71926]: Done uploading segments... Aug 11 14:26:37.109426455 (69h0m31.540506586s) [upload=71926]: Upload finished. err: <nil> Duration: 7.018813472s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.048792998s FAIL (2025-08-11 14:28:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.048815862s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...............X......X................X............X.......] duration: 1.902199022s FAIL (2025-08-11 14:28:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 1.902221264s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [................X......X................X............X......] duration: 1.84546475s FAIL (2025-08-11 14:27:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 1.84549644s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................X..............................X.X...X..X.] duration: 6.925835874s FAIL (2025-08-11 14:27:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after a8a9a4147ef29bc30a9cc6f1443217d7c90a66fe5704ef01df22a945c16fc6b8 != before acdb39bd3ed6066fc55ed4307594725cc9b228bd84f2243570a7e685d7fa13c9 ======LOGS====== Aug 11 14:26:34.154636911 (69h0m28.585717032s) [upload=71926]: Starting upload Aug 11 14:26:34.154683761 (69h0m28.585763892s) [upload=71926]: Done starting upload Aug 11 14:26:34.154779383 (69h0m28.585859515s) [upload=71926]: Uploading segments... Aug 11 14:26:34.154897640 (69h0m28.585977781s) [upload=71926]: Got next segment. Inline: false Aug 11 14:26:34.154948438 (69h0m28.586028569s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:26:34.154952836 (69h0m28.586032967s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:26:34.622954003 (69h0m29.054034124s) [seg_pos={0 0}, upload=71926]: Begin upload segment... Aug 11 14:26:34.622999981 (69h0m29.054080102s) [seg_pos={0 0}, upload=71926]: Done begin upload segment. Aug 11 14:26:34.623018938 (69h0m29.054099069s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623032263 (69h0m29.054112394s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623374248 (69h0m29.054454379s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623638803 (69h0m29.054718934s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623889944 (69h0m29.054970075s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624133850 (69h0m29.055213982s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624515461 (69h0m29.055595592s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624884006 (69h0m29.055964138s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625199660 (69h0m29.056279791s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625528379 (69h0m29.0566085s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625730746 (69h0m29.056810877s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626031792 (69h0m29.057111923s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626240461 (69h0m29.057320593s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626436196 (69h0m29.057516327s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626633714 (69h0m29.057713855s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626823427 (69h0m29.057903558s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627004864 (69h0m29.058084995s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627181031 (69h0m29.058261152s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:35.769773497 (69h0m30.200853628s) [upload=71926]: Next returned nil segment Aug 11 14:26:35.769791913 (69h0m30.200872044s) [upload=71926]: Waiting for error group managing segments... Aug 11 14:26:36.064698127 (69h0m30.495778258s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.134838792 (69h0m30.565918923s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.159959366 (69h0m30.591039497s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.568821428 (69h0m30.999901559s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.599190225 (69h0m31.030270356s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.611299104 (69h0m31.042379225s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.647379849 (69h0m31.07845997s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.665596635 (69h0m31.096676766s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.674096939 (69h0m31.10517707s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.702750032 (69h0m31.133830163s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.709991498 (69h0m31.141071629s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.764409790 (69h0m31.195489921s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.768074693 (69h0m31.199154824s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791039483 (69h0m31.222119614s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791060033 (69h0m31.222140154s) [seg_pos={0 0}, upload=71926]: Segment reached optimal threshold of 14 pieces. Aug 11 14:26:36.791177988 (69h0m31.222258119s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791238053 (69h0m31.222318184s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BwQctDb8mxcmcvf6jvTdQqPx7xv4QMsbaVimioZxGS5vstuvk): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791276257 (69h0m31.222356388s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791374634 (69h0m31.222454765s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aTYNxPbXJGM29Xqovc5VK4cun64wr2DoduaVCvKDj38xCLGKJ): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791427756 (69h0m31.222507887s) [seg_pos={0 0}, upload=71926]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:26:36.791519662 (69h0m31.222599793s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:26:36.791524552 (69h0m31.222604683s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:26:37.109408981 (69h0m31.540489112s) [upload=71926]: Done uploading segments... Aug 11 14:26:37.109426455 (69h0m31.540506586s) [upload=71926]: Upload finished. err: <nil> Duration: 6.92586057s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.336695935s FAIL (2025-08-11 14:27:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.336720981s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [X................X......X................X............X.....] duration: 1.979722833s FAIL (2025-08-11 14:27:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 1.979753311s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X..................X..............................X.X...X..X] duration: 6.887622125s FAIL (2025-08-11 14:26:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after a8a9a4147ef29bc30a9cc6f1443217d7c90a66fe5704ef01df22a945c16fc6b8 != before acdb39bd3ed6066fc55ed4307594725cc9b228bd84f2243570a7e685d7fa13c9 ======LOGS====== Aug 11 14:26:34.154636911 (69h0m28.585717032s) [upload=71926]: Starting upload Aug 11 14:26:34.154683761 (69h0m28.585763892s) [upload=71926]: Done starting upload Aug 11 14:26:34.154779383 (69h0m28.585859515s) [upload=71926]: Uploading segments... Aug 11 14:26:34.154897640 (69h0m28.585977781s) [upload=71926]: Got next segment. Inline: false Aug 11 14:26:34.154948438 (69h0m28.586028569s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:26:34.154952836 (69h0m28.586032967s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:26:34.622954003 (69h0m29.054034124s) [seg_pos={0 0}, upload=71926]: Begin upload segment... Aug 11 14:26:34.622999981 (69h0m29.054080102s) [seg_pos={0 0}, upload=71926]: Done begin upload segment. Aug 11 14:26:34.623018938 (69h0m29.054099069s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623032263 (69h0m29.054112394s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623374248 (69h0m29.054454379s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623638803 (69h0m29.054718934s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.623889944 (69h0m29.054970075s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624133850 (69h0m29.055213982s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624515461 (69h0m29.055595592s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.624884006 (69h0m29.055964138s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625199660 (69h0m29.056279791s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625528379 (69h0m29.0566085s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.625730746 (69h0m29.056810877s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626031792 (69h0m29.057111923s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626240461 (69h0m29.057320593s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626436196 (69h0m29.057516327s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626633714 (69h0m29.057713855s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.626823427 (69h0m29.057903558s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627004864 (69h0m29.058084995s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:34.627181031 (69h0m29.058261152s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Uploading piece... Aug 11 14:26:35.769773497 (69h0m30.200853628s) [upload=71926]: Next returned nil segment Aug 11 14:26:35.769791913 (69h0m30.200872044s) [upload=71926]: Waiting for error group managing segments... Aug 11 14:26:36.064698127 (69h0m30.495778258s) [noise=true , address=64.49.13.155:4913 , piece_id=2X6AIPEAPNBARJPT3JWL5MOQ56UY73XSU32W6KQJLHTEAQSZSEDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.134838792 (69h0m30.565918923s) [noise=true , address=64.49.13.155:4914 , piece_id=CUAUH3PTP75VSRPRLTU7BJXGWA3CBFMEQCNOCAW4SZNI2OYTCAQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.159959366 (69h0m30.591039497s) [noise=true , address=64.49.13.170:4717 , piece_id=NMCU63QH52XTL2BRWNK53EWGC37LVFHIEGBQROVZDUN5J5SOCODQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.568821428 (69h0m30.999901559s) [noise=true , address=77.90.31.12:4605 , piece_id=NTJAJVTM5MJ6O3VMSVEPY3H72DSVJIZNEKS27CPMSMXXCE4FZL2Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.599190225 (69h0m31.030270356s) [noise=true , address=104.36.82.61:4384 , piece_id=PZ2MOGNQOHNO7B3Y6LIY5QPL6YO2KMUEXKVXVM4QLOWKXIE4ZQ7Q, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.611299104 (69h0m31.042379225s) [noise=true , address=104.36.82.19:4432 , piece_id=AQDF5QISTJLNHHKTONLFW5OEXTGVSGNCCR3KSBMFS7T3QZ7MUKVQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.647379849 (69h0m31.07845997s) [noise=true , address=104.36.82.21:4481 , piece_id=I5EEQPZ5AVQWKKXOQ76XE4DKDXRUX56PIDHJAPEVOOKH6D5KUPQA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.665596635 (69h0m31.096676766s) [noise=true , address=104.36.82.35:4581 , piece_id=G5DCBG3UV2FJEHVUAO3L5QUBPXVGKE5XBRQCZ4BABHXLUOMGKYDQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.674096939 (69h0m31.10517707s) [noise=true , address=104.36.82.37:4664 , piece_id=OJPI3PNSJ3B77VYHBUD6QLI24ETS5S7YW3VJN2XMMQ6YB235A7HQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.702750032 (69h0m31.133830163s) [noise=true , address=104.36.82.23:4456 , piece_id=JFEJ5UJZIUA4CMWJV557CTKSIQKXOVNNYOFZ3JAYA7B53UFFLDOQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.709991498 (69h0m31.141071629s) [noise=true , address=104.36.82.60:4361 , piece_id=3XUAWFD57L7SSU3YW7F53K3FKKXZOZMAQAN6X6O3BYPL65XHBTMA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.764409790 (69h0m31.195489921s) [noise=true , address=104.36.82.17:4347 , piece_id=H6RKE4YFS7XXBLYTDD765CTIQ6ZCJPQYAHEWFVORBXCW3XRSW5BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.768074693 (69h0m31.199154824s) [noise=true , address=104.36.82.18:4383 , piece_id=OUABGP2I52FID3WUB6UN7MB4BVMRIO3PNNF6PSWYGGPIYR6XQ3BQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791039483 (69h0m31.222119614s) [noise=true , address=104.36.82.46:4495 , piece_id=JN36UQB7DIKM5RNHBCWO4XAUJRLUH3GYQF655ZRV36HTHCISMT2A, seg_pos={0 0}, upload=71926]: Done uploading piece. err: <nil> Aug 11 14:26:36.791060033 (69h0m31.222140154s) [seg_pos={0 0}, upload=71926]: Segment reached optimal threshold of 14 pieces. Aug 11 14:26:36.791177988 (69h0m31.222258119s) [noise=true , address=104.36.82.44:4899 , piece_id=5HYWZA6CQQEQNDMEJPOG4GKO2NASSZI4VHGOM7XWF4JZ7NQRKKQQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791238053 (69h0m31.222318184s) [noise=true , address=104.36.82.17:4339 , piece_id=UAGJCP7UPX6KD6MQTGLQ23I6VIBNB6TT5IJ5EWQDLOZVXFQMX4IA, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BwQctDb8mxcmcvf6jvTdQqPx7xv4QMsbaVimioZxGS5vstuvk): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791276257 (69h0m31.222356388s) [noise=true , address=104.36.82.38:4690 , piece_id=EDSB5QND63NBNT3XKBLC66O6DBMV5GHUDLBYFEVGOJQWQH3SKAKQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791374634 (69h0m31.222454765s) [noise=true , address=104.36.82.16:4325 , piece_id=XKAP2NYBDNVJSHCIWUK4MYI3CHUBV2NLMXZTXNTVTDRQQGK7AVCQ, seg_pos={0 0}, upload=71926]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aTYNxPbXJGM29Xqovc5VK4cun64wr2DoduaVCvKDj38xCLGKJ): protocol: expected piece hash; context canceled; EOF Aug 11 14:26:36.791427756 (69h0m31.222507887s) [seg_pos={0 0}, upload=71926]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:26:36.791519662 (69h0m31.222599793s) [upload=71926]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:26:36.791524552 (69h0m31.222604683s) [upload=71926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:26:37.109408981 (69h0m31.540489112s) [upload=71926]: Done uploading segments... Aug 11 14:26:37.109426455 (69h0m31.540506586s) [upload=71926]: Upload finished. err: <nil> Duration: 6.887646343s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.287633149s FAIL (2025-08-11 14:26:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.287650481s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X..................X..............................X.X...X..] duration: 6.473310527s FAIL (2025-08-11 14:26:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 14:24:55.941093126 (68h58m50.372173287s) [upload=71894]: Starting upload Aug 11 14:24:55.941159512 (68h58m50.372239683s) [upload=71894]: Done starting upload Aug 11 14:24:55.941295122 (68h58m50.372375283s) [upload=71894]: Uploading segments... Aug 11 14:24:55.941601458 (68h58m50.372681589s) [upload=71894]: Got next segment. Inline: false Aug 11 14:24:55.941644972 (68h58m50.372725113s) [upload=71894]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:24:55.941651604 (68h58m50.372731765s) [upload=71894]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:24:56.403814559 (68h58m50.83489469s) [seg_pos={0 0}, upload=71894]: Begin upload segment... Aug 11 14:24:56.403857250 (68h58m50.834937381s) [seg_pos={0 0}, upload=71894]: Done begin upload segment. Aug 11 14:24:56.403876898 (68h58m50.834957029s) [noise=true , address=64.49.13.155:4915 , piece_id=ZM5WSQX5VUT5DRSZ2JC4LE366NN2JGFM23EYDBHANNBJ7CN4T5WA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.403936051 (68h58m50.835016362s) [noise=true , address=104.36.82.10:4092 , piece_id=RJJEMUL2UMQ5JKFQKV6OJ4HD4OVJOXO4TFAPEHCSZ3FTR63JYGJQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404223972 (68h58m50.835304104s) [noise=true , address=104.36.82.14:4224 , piece_id=V64DVMG7BSBFRBGVBQSIWEC7LYLP4T3P5G7AVETHTZHMMBKGQUPQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404392064 (68h58m50.835472185s) [noise=true , address=104.36.82.60:4365 , piece_id=W7K7KT4GMT6QBWRKUAPYP7BKDWF3BOTYKH7V3MOOGM52LRN5MT5Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404542682 (68h58m50.835622813s) [noise=true , address=104.36.82.23:4467 , piece_id=3VX6R3WJBB2QZAPKDPYPBJ3OOVYJTEFOEV3OXTRDRRL2AJKRNJGA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404677249 (68h58m50.83575738s) [noise=true , address=104.36.82.59:4329 , piece_id=BN5NUKMWJCIYCWYUEWCIMUFZGWJNDNNW6LM7E5YNGAGCVHNJG3BA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404799193 (68h58m50.835879324s) [noise=true , address=104.36.82.5:4855 , piece_id=TA5LDUYYPNNRA25QJLCEURXQTZL3Q6IOEIOM3KW43GXEZZXBQO5Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405081413 (68h58m50.836161544s) [noise=true , address=104.36.82.47:4547 , piece_id=UNULQQ6CUKV6VAM2Z4G32A2RV7EMYOQDZM4GKS7L35UDHT6RGFOQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405106381 (68h58m50.836186512s) [noise=true , address=104.36.82.15:4279 , piece_id=GNREOPWOW33F3SGM457PREYHVXO37RCD4STD2FNHI7Q6ZQF7F4ZQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405382970 (68h58m50.836463101s) [noise=true , address=104.36.82.6:4906 , piece_id=6Y5OASMSXYS2KA3Q65WAHUO2J54U3ZY7RERZCVNBSUGBTU7S7VQQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405416874 (68h58m50.836497006s) [noise=true , address=104.36.82.61:4384 , piece_id=CV7D2P37HVJLXUUUYZYBBVE3HADXDFKMW3ZQN3L2DEQSXUSXXAXA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405447473 (68h58m50.836527604s) [noise=true , address=104.36.82.4:4817 , piece_id=KCHQ3YA7EIH4ECAO3R3TK77NC7HRYY2JFDKMCZP7774NXTKJRC3Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405689777 (68h58m50.836769898s) [noise=true , address=64.49.13.169:4709 , piece_id=HSVEVYZ3LHTV7UO7M6N4U4K2YZIYFWHNQCILIIV3ETOUYNDWHDCQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405731105 (68h58m50.836811236s) [noise=true , address=104.36.82.61:4374 , piece_id=OQO3DQQYGJN2WT4AT7I66DOMAQOERLGZVAI3M3NGAVPOHDZUHF5A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405990632 (68h58m50.837070753s) [noise=true , address=104.36.82.45:4934 , piece_id=GFJOZJI2EFEYFVWR644WTWTCYONMTH6PTF5HKEAJFA66QV2JS73A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406003687 (68h58m50.837083818s) [noise=true , address=104.36.82.60:4367 , piece_id=AQOIGGH55JSIMZDALXBCSIB4HNSXC76UOQE6IL6PKMOFXPDIUUTA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406030769 (68h58m50.8371109s) [noise=true , address=104.36.82.54:4191 , piece_id=KUV6UO7VVLGGMGORKNERBM7JALLRPTSYEVFWNL4UOQ3WID3WI55A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406255890 (68h58m50.837336021s) [noise=true , address=104.36.82.34:4559 , piece_id=H67XFI373HPW65LFRBS74LD3TC4DU2SOOPMBL6IMCSEJ3CIO22RA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:57.600022080 (68h58m52.031102221s) [upload=71894]: Next returned nil segment Aug 11 14:24:57.600088878 (68h58m52.031169009s) [upload=71894]: Waiting for error group managing segments... Aug 11 14:24:57.900729874 (68h58m52.331810005s) [noise=true , address=64.49.13.169:4709 , piece_id=HSVEVYZ3LHTV7UO7M6N4U4K2YZIYFWHNQCILIIV3ETOUYNDWHDCQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:57.997000845 (68h58m52.428080966s) [noise=true , address=64.49.13.155:4915 , piece_id=ZM5WSQX5VUT5DRSZ2JC4LE366NN2JGFM23EYDBHANNBJ7CN4T5WA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.471891735 (68h58m52.902971866s) [noise=true , address=104.36.82.54:4191 , piece_id=KUV6UO7VVLGGMGORKNERBM7JALLRPTSYEVFWNL4UOQ3WID3WI55A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.477520455 (68h58m52.908600586s) [noise=true , address=104.36.82.14:4224 , piece_id=V64DVMG7BSBFRBGVBQSIWEC7LYLP4T3P5G7AVETHTZHMMBKGQUPQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.480149785 (68h58m52.911229916s) [noise=true , address=104.36.82.4:4817 , piece_id=KCHQ3YA7EIH4ECAO3R3TK77NC7HRYY2JFDKMCZP7774NXTKJRC3Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.587096173 (68h58m53.018176304s) [noise=true , address=104.36.82.45:4934 , piece_id=GFJOZJI2EFEYFVWR644WTWTCYONMTH6PTF5HKEAJFA66QV2JS73A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.634664799 (68h58m53.065744941s) [noise=true , address=104.36.82.6:4906 , piece_id=6Y5OASMSXYS2KA3Q65WAHUO2J54U3ZY7RERZCVNBSUGBTU7S7VQQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.707247797 (68h58m53.138327928s) [noise=true , address=104.36.82.23:4467 , piece_id=3VX6R3WJBB2QZAPKDPYPBJ3OOVYJTEFOEV3OXTRDRRL2AJKRNJGA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.723422966 (68h58m53.154503097s) [noise=true , address=104.36.82.47:4547 , piece_id=UNULQQ6CUKV6VAM2Z4G32A2RV7EMYOQDZM4GKS7L35UDHT6RGFOQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.797481658 (68h58m53.228561789s) [noise=true , address=104.36.82.59:4329 , piece_id=BN5NUKMWJCIYCWYUEWCIMUFZGWJNDNNW6LM7E5YNGAGCVHNJG3BA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.935000611 (68h58m53.366080732s) [noise=true , address=104.36.82.15:4279 , piece_id=GNREOPWOW33F3SGM457PREYHVXO37RCD4STD2FNHI7Q6ZQF7F4ZQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.980825551 (68h58m53.411905682s) [noise=true , address=104.36.82.61:4374 , piece_id=OQO3DQQYGJN2WT4AT7I66DOMAQOERLGZVAI3M3NGAVPOHDZUHF5A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.112844522 (68h58m53.543924653s) [noise=true , address=104.36.82.5:4855 , piece_id=TA5LDUYYPNNRA25QJLCEURXQTZL3Q6IOEIOM3KW43GXEZZXBQO5Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.191085556 (68h58m53.622165687s) [noise=true , address=104.36.82.61:4384 , piece_id=CV7D2P37HVJLXUUUYZYBBVE3HADXDFKMW3ZQN3L2DEQSXUSXXAXA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.191102147 (68h58m53.622182278s) [seg_pos={0 0}, upload=71894]: Segment reached optimal threshold of 14 pieces. Aug 11 14:24:59.191338870 (68h58m53.622419001s) [noise=true , address=104.36.82.34:4559 , piece_id=H67XFI373HPW65LFRBS74LD3TC4DU2SOOPMBL6IMCSEJ3CIO22RA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXGQfMeXYCUuSh5TUe8pTwwd19wn68r9buS11N8aJdx6vZysf): protocol: expected piece hash; context canceled; EOF Aug 11 14:24:59.191368838 (68h58m53.622448969s) [noise=true , address=104.36.82.60:4367 , piece_id=AQOIGGH55JSIMZDALXBCSIB4HNSXC76UOQE6IL6PKMOFXPDIUUTA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Aug 11 14:24:59.191407531 (68h58m53.622487652s) [noise=true , address=104.36.82.10:4092 , piece_id=RJJEMUL2UMQ5JKFQKV6OJ4HD4OVJOXO4TFAPEHCSZ3FTR63JYGJQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection; write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection Aug 11 14:24:59.191507192 (68h58m53.622587313s) [noise=true , address=104.36.82.60:4365 , piece_id=W7K7KT4GMT6QBWRKUAPYP7BKDWF3BOTYKH7V3MOOGM52LRN5MT5Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a): protocol: write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection; write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection Aug 11 14:24:59.191533393 (68h58m53.622613524s) [seg_pos={0 0}, upload=71894]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:24:59.191612494 (68h58m53.622692625s) [upload=71894]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:24:59.191616942 (68h58m53.622697073s) [upload=71894]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:24:59.604892449 (68h58m54.03597258s) [upload=71894]: Done uploading segments... Aug 11 14:24:59.604912007 (68h58m54.035992138s) [upload=71894]: Upload finished. err: <nil> Duration: 6.473331797s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.X................X......X................X............X....] duration: 2.061453815s FAIL (2025-08-11 14:26:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 2.061496085s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X..................X..............................X.X...X.] duration: 7.017850834s FAIL (2025-08-11 14:25:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 14:24:55.941093126 (68h58m50.372173287s) [upload=71894]: Starting upload Aug 11 14:24:55.941159512 (68h58m50.372239683s) [upload=71894]: Done starting upload Aug 11 14:24:55.941295122 (68h58m50.372375283s) [upload=71894]: Uploading segments... Aug 11 14:24:55.941601458 (68h58m50.372681589s) [upload=71894]: Got next segment. Inline: false Aug 11 14:24:55.941644972 (68h58m50.372725113s) [upload=71894]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:24:55.941651604 (68h58m50.372731765s) [upload=71894]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:24:56.403814559 (68h58m50.83489469s) [seg_pos={0 0}, upload=71894]: Begin upload segment... Aug 11 14:24:56.403857250 (68h58m50.834937381s) [seg_pos={0 0}, upload=71894]: Done begin upload segment. Aug 11 14:24:56.403876898 (68h58m50.834957029s) [noise=true , address=64.49.13.155:4915 , piece_id=ZM5WSQX5VUT5DRSZ2JC4LE366NN2JGFM23EYDBHANNBJ7CN4T5WA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.403936051 (68h58m50.835016362s) [noise=true , address=104.36.82.10:4092 , piece_id=RJJEMUL2UMQ5JKFQKV6OJ4HD4OVJOXO4TFAPEHCSZ3FTR63JYGJQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404223972 (68h58m50.835304104s) [noise=true , address=104.36.82.14:4224 , piece_id=V64DVMG7BSBFRBGVBQSIWEC7LYLP4T3P5G7AVETHTZHMMBKGQUPQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404392064 (68h58m50.835472185s) [noise=true , address=104.36.82.60:4365 , piece_id=W7K7KT4GMT6QBWRKUAPYP7BKDWF3BOTYKH7V3MOOGM52LRN5MT5Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404542682 (68h58m50.835622813s) [noise=true , address=104.36.82.23:4467 , piece_id=3VX6R3WJBB2QZAPKDPYPBJ3OOVYJTEFOEV3OXTRDRRL2AJKRNJGA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404677249 (68h58m50.83575738s) [noise=true , address=104.36.82.59:4329 , piece_id=BN5NUKMWJCIYCWYUEWCIMUFZGWJNDNNW6LM7E5YNGAGCVHNJG3BA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404799193 (68h58m50.835879324s) [noise=true , address=104.36.82.5:4855 , piece_id=TA5LDUYYPNNRA25QJLCEURXQTZL3Q6IOEIOM3KW43GXEZZXBQO5Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405081413 (68h58m50.836161544s) [noise=true , address=104.36.82.47:4547 , piece_id=UNULQQ6CUKV6VAM2Z4G32A2RV7EMYOQDZM4GKS7L35UDHT6RGFOQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405106381 (68h58m50.836186512s) [noise=true , address=104.36.82.15:4279 , piece_id=GNREOPWOW33F3SGM457PREYHVXO37RCD4STD2FNHI7Q6ZQF7F4ZQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405382970 (68h58m50.836463101s) [noise=true , address=104.36.82.6:4906 , piece_id=6Y5OASMSXYS2KA3Q65WAHUO2J54U3ZY7RERZCVNBSUGBTU7S7VQQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405416874 (68h58m50.836497006s) [noise=true , address=104.36.82.61:4384 , piece_id=CV7D2P37HVJLXUUUYZYBBVE3HADXDFKMW3ZQN3L2DEQSXUSXXAXA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405447473 (68h58m50.836527604s) [noise=true , address=104.36.82.4:4817 , piece_id=KCHQ3YA7EIH4ECAO3R3TK77NC7HRYY2JFDKMCZP7774NXTKJRC3Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405689777 (68h58m50.836769898s) [noise=true , address=64.49.13.169:4709 , piece_id=HSVEVYZ3LHTV7UO7M6N4U4K2YZIYFWHNQCILIIV3ETOUYNDWHDCQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405731105 (68h58m50.836811236s) [noise=true , address=104.36.82.61:4374 , piece_id=OQO3DQQYGJN2WT4AT7I66DOMAQOERLGZVAI3M3NGAVPOHDZUHF5A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405990632 (68h58m50.837070753s) [noise=true , address=104.36.82.45:4934 , piece_id=GFJOZJI2EFEYFVWR644WTWTCYONMTH6PTF5HKEAJFA66QV2JS73A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406003687 (68h58m50.837083818s) [noise=true , address=104.36.82.60:4367 , piece_id=AQOIGGH55JSIMZDALXBCSIB4HNSXC76UOQE6IL6PKMOFXPDIUUTA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406030769 (68h58m50.8371109s) [noise=true , address=104.36.82.54:4191 , piece_id=KUV6UO7VVLGGMGORKNERBM7JALLRPTSYEVFWNL4UOQ3WID3WI55A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406255890 (68h58m50.837336021s) [noise=true , address=104.36.82.34:4559 , piece_id=H67XFI373HPW65LFRBS74LD3TC4DU2SOOPMBL6IMCSEJ3CIO22RA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:57.600022080 (68h58m52.031102221s) [upload=71894]: Next returned nil segment Aug 11 14:24:57.600088878 (68h58m52.031169009s) [upload=71894]: Waiting for error group managing segments... Aug 11 14:24:57.900729874 (68h58m52.331810005s) [noise=true , address=64.49.13.169:4709 , piece_id=HSVEVYZ3LHTV7UO7M6N4U4K2YZIYFWHNQCILIIV3ETOUYNDWHDCQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:57.997000845 (68h58m52.428080966s) [noise=true , address=64.49.13.155:4915 , piece_id=ZM5WSQX5VUT5DRSZ2JC4LE366NN2JGFM23EYDBHANNBJ7CN4T5WA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.471891735 (68h58m52.902971866s) [noise=true , address=104.36.82.54:4191 , piece_id=KUV6UO7VVLGGMGORKNERBM7JALLRPTSYEVFWNL4UOQ3WID3WI55A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.477520455 (68h58m52.908600586s) [noise=true , address=104.36.82.14:4224 , piece_id=V64DVMG7BSBFRBGVBQSIWEC7LYLP4T3P5G7AVETHTZHMMBKGQUPQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.480149785 (68h58m52.911229916s) [noise=true , address=104.36.82.4:4817 , piece_id=KCHQ3YA7EIH4ECAO3R3TK77NC7HRYY2JFDKMCZP7774NXTKJRC3Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.587096173 (68h58m53.018176304s) [noise=true , address=104.36.82.45:4934 , piece_id=GFJOZJI2EFEYFVWR644WTWTCYONMTH6PTF5HKEAJFA66QV2JS73A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.634664799 (68h58m53.065744941s) [noise=true , address=104.36.82.6:4906 , piece_id=6Y5OASMSXYS2KA3Q65WAHUO2J54U3ZY7RERZCVNBSUGBTU7S7VQQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.707247797 (68h58m53.138327928s) [noise=true , address=104.36.82.23:4467 , piece_id=3VX6R3WJBB2QZAPKDPYPBJ3OOVYJTEFOEV3OXTRDRRL2AJKRNJGA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.723422966 (68h58m53.154503097s) [noise=true , address=104.36.82.47:4547 , piece_id=UNULQQ6CUKV6VAM2Z4G32A2RV7EMYOQDZM4GKS7L35UDHT6RGFOQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.797481658 (68h58m53.228561789s) [noise=true , address=104.36.82.59:4329 , piece_id=BN5NUKMWJCIYCWYUEWCIMUFZGWJNDNNW6LM7E5YNGAGCVHNJG3BA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.935000611 (68h58m53.366080732s) [noise=true , address=104.36.82.15:4279 , piece_id=GNREOPWOW33F3SGM457PREYHVXO37RCD4STD2FNHI7Q6ZQF7F4ZQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.980825551 (68h58m53.411905682s) [noise=true , address=104.36.82.61:4374 , piece_id=OQO3DQQYGJN2WT4AT7I66DOMAQOERLGZVAI3M3NGAVPOHDZUHF5A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.112844522 (68h58m53.543924653s) [noise=true , address=104.36.82.5:4855 , piece_id=TA5LDUYYPNNRA25QJLCEURXQTZL3Q6IOEIOM3KW43GXEZZXBQO5Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.191085556 (68h58m53.622165687s) [noise=true , address=104.36.82.61:4384 , piece_id=CV7D2P37HVJLXUUUYZYBBVE3HADXDFKMW3ZQN3L2DEQSXUSXXAXA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.191102147 (68h58m53.622182278s) [seg_pos={0 0}, upload=71894]: Segment reached optimal threshold of 14 pieces. Aug 11 14:24:59.191338870 (68h58m53.622419001s) [noise=true , address=104.36.82.34:4559 , piece_id=H67XFI373HPW65LFRBS74LD3TC4DU2SOOPMBL6IMCSEJ3CIO22RA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXGQfMeXYCUuSh5TUe8pTwwd19wn68r9buS11N8aJdx6vZysf): protocol: expected piece hash; context canceled; EOF Aug 11 14:24:59.191368838 (68h58m53.622448969s) [noise=true , address=104.36.82.60:4367 , piece_id=AQOIGGH55JSIMZDALXBCSIB4HNSXC76UOQE6IL6PKMOFXPDIUUTA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Aug 11 14:24:59.191407531 (68h58m53.622487652s) [noise=true , address=104.36.82.10:4092 , piece_id=RJJEMUL2UMQ5JKFQKV6OJ4HD4OVJOXO4TFAPEHCSZ3FTR63JYGJQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection; write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection Aug 11 14:24:59.191507192 (68h58m53.622587313s) [noise=true , address=104.36.82.60:4365 , piece_id=W7K7KT4GMT6QBWRKUAPYP7BKDWF3BOTYKH7V3MOOGM52LRN5MT5Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a): protocol: write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection; write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection Aug 11 14:24:59.191533393 (68h58m53.622613524s) [seg_pos={0 0}, upload=71894]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:24:59.191612494 (68h58m53.622692625s) [upload=71894]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:24:59.191616942 (68h58m53.622697073s) [upload=71894]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:24:59.604892449 (68h58m54.03597258s) [upload=71894]: Done uploading segments... Aug 11 14:24:59.604912007 (68h58m54.035992138s) [upload=71894]: Upload finished. err: <nil> Duration: 7.01788002s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 7.469364164s FAIL (2025-08-11 14:25:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.469387708s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..X................X......X................X............X...] duration: 1.955635723s FAIL (2025-08-11 14:25:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 1.955667122s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X..................X..............................X.X...X] duration: 5.00961903s FAIL (2025-08-11 14:25:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 14:24:55.941093126 (68h58m50.372173287s) [upload=71894]: Starting upload Aug 11 14:24:55.941159512 (68h58m50.372239683s) [upload=71894]: Done starting upload Aug 11 14:24:55.941295122 (68h58m50.372375283s) [upload=71894]: Uploading segments... Aug 11 14:24:55.941601458 (68h58m50.372681589s) [upload=71894]: Got next segment. Inline: false Aug 11 14:24:55.941644972 (68h58m50.372725113s) [upload=71894]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:24:55.941651604 (68h58m50.372731765s) [upload=71894]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:24:56.403814559 (68h58m50.83489469s) [seg_pos={0 0}, upload=71894]: Begin upload segment... Aug 11 14:24:56.403857250 (68h58m50.834937381s) [seg_pos={0 0}, upload=71894]: Done begin upload segment. Aug 11 14:24:56.403876898 (68h58m50.834957029s) [noise=true , address=64.49.13.155:4915 , piece_id=ZM5WSQX5VUT5DRSZ2JC4LE366NN2JGFM23EYDBHANNBJ7CN4T5WA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.403936051 (68h58m50.835016362s) [noise=true , address=104.36.82.10:4092 , piece_id=RJJEMUL2UMQ5JKFQKV6OJ4HD4OVJOXO4TFAPEHCSZ3FTR63JYGJQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404223972 (68h58m50.835304104s) [noise=true , address=104.36.82.14:4224 , piece_id=V64DVMG7BSBFRBGVBQSIWEC7LYLP4T3P5G7AVETHTZHMMBKGQUPQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404392064 (68h58m50.835472185s) [noise=true , address=104.36.82.60:4365 , piece_id=W7K7KT4GMT6QBWRKUAPYP7BKDWF3BOTYKH7V3MOOGM52LRN5MT5Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404542682 (68h58m50.835622813s) [noise=true , address=104.36.82.23:4467 , piece_id=3VX6R3WJBB2QZAPKDPYPBJ3OOVYJTEFOEV3OXTRDRRL2AJKRNJGA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404677249 (68h58m50.83575738s) [noise=true , address=104.36.82.59:4329 , piece_id=BN5NUKMWJCIYCWYUEWCIMUFZGWJNDNNW6LM7E5YNGAGCVHNJG3BA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.404799193 (68h58m50.835879324s) [noise=true , address=104.36.82.5:4855 , piece_id=TA5LDUYYPNNRA25QJLCEURXQTZL3Q6IOEIOM3KW43GXEZZXBQO5Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405081413 (68h58m50.836161544s) [noise=true , address=104.36.82.47:4547 , piece_id=UNULQQ6CUKV6VAM2Z4G32A2RV7EMYOQDZM4GKS7L35UDHT6RGFOQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405106381 (68h58m50.836186512s) [noise=true , address=104.36.82.15:4279 , piece_id=GNREOPWOW33F3SGM457PREYHVXO37RCD4STD2FNHI7Q6ZQF7F4ZQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405382970 (68h58m50.836463101s) [noise=true , address=104.36.82.6:4906 , piece_id=6Y5OASMSXYS2KA3Q65WAHUO2J54U3ZY7RERZCVNBSUGBTU7S7VQQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405416874 (68h58m50.836497006s) [noise=true , address=104.36.82.61:4384 , piece_id=CV7D2P37HVJLXUUUYZYBBVE3HADXDFKMW3ZQN3L2DEQSXUSXXAXA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405447473 (68h58m50.836527604s) [noise=true , address=104.36.82.4:4817 , piece_id=KCHQ3YA7EIH4ECAO3R3TK77NC7HRYY2JFDKMCZP7774NXTKJRC3Q, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405689777 (68h58m50.836769898s) [noise=true , address=64.49.13.169:4709 , piece_id=HSVEVYZ3LHTV7UO7M6N4U4K2YZIYFWHNQCILIIV3ETOUYNDWHDCQ, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405731105 (68h58m50.836811236s) [noise=true , address=104.36.82.61:4374 , piece_id=OQO3DQQYGJN2WT4AT7I66DOMAQOERLGZVAI3M3NGAVPOHDZUHF5A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.405990632 (68h58m50.837070753s) [noise=true , address=104.36.82.45:4934 , piece_id=GFJOZJI2EFEYFVWR644WTWTCYONMTH6PTF5HKEAJFA66QV2JS73A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406003687 (68h58m50.837083818s) [noise=true , address=104.36.82.60:4367 , piece_id=AQOIGGH55JSIMZDALXBCSIB4HNSXC76UOQE6IL6PKMOFXPDIUUTA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406030769 (68h58m50.8371109s) [noise=true , address=104.36.82.54:4191 , piece_id=KUV6UO7VVLGGMGORKNERBM7JALLRPTSYEVFWNL4UOQ3WID3WI55A, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:56.406255890 (68h58m50.837336021s) [noise=true , address=104.36.82.34:4559 , piece_id=H67XFI373HPW65LFRBS74LD3TC4DU2SOOPMBL6IMCSEJ3CIO22RA, seg_pos={0 0}, upload=71894]: Uploading piece... Aug 11 14:24:57.600022080 (68h58m52.031102221s) [upload=71894]: Next returned nil segment Aug 11 14:24:57.600088878 (68h58m52.031169009s) [upload=71894]: Waiting for error group managing segments... Aug 11 14:24:57.900729874 (68h58m52.331810005s) [noise=true , address=64.49.13.169:4709 , piece_id=HSVEVYZ3LHTV7UO7M6N4U4K2YZIYFWHNQCILIIV3ETOUYNDWHDCQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:57.997000845 (68h58m52.428080966s) [noise=true , address=64.49.13.155:4915 , piece_id=ZM5WSQX5VUT5DRSZ2JC4LE366NN2JGFM23EYDBHANNBJ7CN4T5WA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.471891735 (68h58m52.902971866s) [noise=true , address=104.36.82.54:4191 , piece_id=KUV6UO7VVLGGMGORKNERBM7JALLRPTSYEVFWNL4UOQ3WID3WI55A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.477520455 (68h58m52.908600586s) [noise=true , address=104.36.82.14:4224 , piece_id=V64DVMG7BSBFRBGVBQSIWEC7LYLP4T3P5G7AVETHTZHMMBKGQUPQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.480149785 (68h58m52.911229916s) [noise=true , address=104.36.82.4:4817 , piece_id=KCHQ3YA7EIH4ECAO3R3TK77NC7HRYY2JFDKMCZP7774NXTKJRC3Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.587096173 (68h58m53.018176304s) [noise=true , address=104.36.82.45:4934 , piece_id=GFJOZJI2EFEYFVWR644WTWTCYONMTH6PTF5HKEAJFA66QV2JS73A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.634664799 (68h58m53.065744941s) [noise=true , address=104.36.82.6:4906 , piece_id=6Y5OASMSXYS2KA3Q65WAHUO2J54U3ZY7RERZCVNBSUGBTU7S7VQQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.707247797 (68h58m53.138327928s) [noise=true , address=104.36.82.23:4467 , piece_id=3VX6R3WJBB2QZAPKDPYPBJ3OOVYJTEFOEV3OXTRDRRL2AJKRNJGA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.723422966 (68h58m53.154503097s) [noise=true , address=104.36.82.47:4547 , piece_id=UNULQQ6CUKV6VAM2Z4G32A2RV7EMYOQDZM4GKS7L35UDHT6RGFOQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.797481658 (68h58m53.228561789s) [noise=true , address=104.36.82.59:4329 , piece_id=BN5NUKMWJCIYCWYUEWCIMUFZGWJNDNNW6LM7E5YNGAGCVHNJG3BA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.935000611 (68h58m53.366080732s) [noise=true , address=104.36.82.15:4279 , piece_id=GNREOPWOW33F3SGM457PREYHVXO37RCD4STD2FNHI7Q6ZQF7F4ZQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:58.980825551 (68h58m53.411905682s) [noise=true , address=104.36.82.61:4374 , piece_id=OQO3DQQYGJN2WT4AT7I66DOMAQOERLGZVAI3M3NGAVPOHDZUHF5A, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.112844522 (68h58m53.543924653s) [noise=true , address=104.36.82.5:4855 , piece_id=TA5LDUYYPNNRA25QJLCEURXQTZL3Q6IOEIOM3KW43GXEZZXBQO5Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.191085556 (68h58m53.622165687s) [noise=true , address=104.36.82.61:4384 , piece_id=CV7D2P37HVJLXUUUYZYBBVE3HADXDFKMW3ZQN3L2DEQSXUSXXAXA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: <nil> Aug 11 14:24:59.191102147 (68h58m53.622182278s) [seg_pos={0 0}, upload=71894]: Segment reached optimal threshold of 14 pieces. Aug 11 14:24:59.191338870 (68h58m53.622419001s) [noise=true , address=104.36.82.34:4559 , piece_id=H67XFI373HPW65LFRBS74LD3TC4DU2SOOPMBL6IMCSEJ3CIO22RA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXGQfMeXYCUuSh5TUe8pTwwd19wn68r9buS11N8aJdx6vZysf): protocol: expected piece hash; context canceled; EOF Aug 11 14:24:59.191368838 (68h58m53.622448969s) [noise=true , address=104.36.82.60:4367 , piece_id=AQOIGGH55JSIMZDALXBCSIB4HNSXC76UOQE6IL6PKMOFXPDIUUTA, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Aug 11 14:24:59.191407531 (68h58m53.622487652s) [noise=true , address=104.36.82.10:4092 , piece_id=RJJEMUL2UMQ5JKFQKV6OJ4HD4OVJOXO4TFAPEHCSZ3FTR63JYGJQ, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128P9omT47U7tm1k4XdeUR5vd1VAqskCvyfjF6VDy1ncwMDoZm9): protocol: write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection; write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53456->104.36.82.10:4092: use of closed network connection Aug 11 14:24:59.191507192 (68h58m53.622587313s) [noise=true , address=104.36.82.60:4365 , piece_id=W7K7KT4GMT6QBWRKUAPYP7BKDWF3BOTYKH7V3MOOGM52LRN5MT5Q, seg_pos={0 0}, upload=71894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a): protocol: write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection; write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57626->104.36.82.60:4365: use of closed network connection Aug 11 14:24:59.191533393 (68h58m53.622613524s) [seg_pos={0 0}, upload=71894]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:24:59.191612494 (68h58m53.622692625s) [upload=71894]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:24:59.191616942 (68h58m53.622697073s) [upload=71894]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:24:59.604892449 (68h58m54.03597258s) [upload=71894]: Done uploading segments... Aug 11 14:24:59.604912007 (68h58m54.035992138s) [upload=71894]: Upload finished. err: <nil> Duration: 5.009640974s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.346623442s FAIL (2025-08-11 14:24:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.346644633s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...X................X......X................X............X..] duration: 2.007448491s FAIL (2025-08-11 14:24:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 2.007474402s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X..................X..............................X.X...] duration: 7.512665705s FAIL (2025-08-11 14:24:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 14:22:11.588492069 (68h56m6.0195722s) [upload=71849]: Starting upload Aug 11 14:22:11.588534811 (68h56m6.019614932s) [upload=71849]: Done starting upload Aug 11 14:22:11.588641926 (68h56m6.019722047s) [upload=71849]: Uploading segments... Aug 11 14:22:11.588701349 (68h56m6.01978148s) [upload=71849]: Got next segment. Inline: false Aug 11 14:22:11.588729163 (68h56m6.019809294s) [upload=71849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:11.588733110 (68h56m6.019813241s) [upload=71849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:22:12.047436437 (68h56m6.478516568s) [seg_pos={0 0}, upload=71849]: Begin upload segment... Aug 11 14:22:12.047509667 (68h56m6.478589828s) [seg_pos={0 0}, upload=71849]: Done begin upload segment. Aug 11 14:22:12.047525597 (68h56m6.478605728s) [noise=true , address=104.36.82.60:4360 , piece_id=CSSK655RYUWKXZP646LKO3BBPHSGBWPSB2QMAI4LKHR7SPC2PGRQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.047541838 (68h56m6.478621969s) [noise=true , address=104.36.82.47:4549 , piece_id=MJQPFWASTOO2EYBJKKQQQUEZBZIE5ZUIUEQUMM5QLPR2LJTIXLRA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.048025404 (68h56m6.479105535s) [noise=true , address=64.49.13.154:4933 , piece_id=GYGTGPN2MG5ETRMPZSVWE54O5F56GGWULDZJFZFESE4LTZWMSPQQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.048991011 (68h56m6.480071142s) [noise=true , address=104.36.82.21:4489 , piece_id=URWEO4323RZN6YV6UOCK4PC3UXIKMQIEOZLCL24FQ33DXKCIKOOQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.049350639 (68h56m6.48043078s) [noise=true , address=104.36.82.17:4339 , piece_id=5YHVPEINCZVUQU75Z3PON57QRECP6LADNQZTHPQMLE7ZFH24AKNQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.049695530 (68h56m6.480775661s) [noise=true , address=77.90.31.13:4643 , piece_id=7ITQU24YQWV5474J5K3ZZNKAEGRQ6FHCYB4EW2BCDPKDAIXKIP6A, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050032955 (68h56m6.481113096s) [noise=true , address=104.36.82.8:4999 , piece_id=CXT4YR5NWU4NBIXPBHNIRTVOW5LZYFADEVVLGFIGQXYWPS772M7Q, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050304054 (68h56m6.481384175s) [noise=true , address=104.36.82.37:4674 , piece_id=5IUTA2L4TCVI73IPYPSTDENQNFX2N4ZSF7EMEBQRAWDL2ARKSUPQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050622062 (68h56m6.481702183s) [noise=true , address=104.36.82.39:4739 , piece_id=B5QJU4VPBYUK5247SR6SCNTBU42Q3CBDK4Z3HGHNLAL76MGAENHA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050944299 (68h56m6.48202443s) [noise=true , address=104.36.82.22:4537 , piece_id=SIWG45D42XCBWWLVCUUQQH3X2AINV3RAJYJ7J3NRICU7A5PLIO6A, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051263940 (68h56m6.482344072s) [noise=true , address=104.36.82.44:4871 , piece_id=ASZ6X7K4BHSYQID47L6AFWZWA5WC43MRMK4CZPUYQLL4EDRAU6WQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051533416 (68h56m6.482613547s) [noise=true , address=104.36.82.35:4594 , piece_id=K2WAYZOFR5IWKKGIGV6QM6C62E47RR7NGCX47ZP4H4QFTOXLTW2Q, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051831166 (68h56m6.482911297s) [noise=true , address=104.36.82.56:4255 , piece_id=XTYNJXNDMWUXIXALMID4QZ3KTC4RYHOHUUEHC3GRE23PWXFO2QXQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052168281 (68h56m6.483248402s) [noise=true , address=64.49.13.170:4721 , piece_id=KDZC5YVY3TRAUWYDHJAVQWMANLPNFKFAO4SCUDK5XVVHUEEBY5WQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052226993 (68h56m6.483307124s) [noise=true , address=104.36.82.62:4417 , piece_id=NNEGV2OM54WCS4S7XSXRXLSK4IJHYGHLT26TVNUZQDA67JWDTAAA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052613883 (68h56m6.483694014s) [noise=true , address=104.36.82.18:4380 , piece_id=QD2IEZ7YMUHDA7P7E5THL6LDKS7KJTXWBVSJUUUFZY5H4SZZCTAQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052921051 (68h56m6.484001182s) [noise=true , address=104.36.82.58:4272 , piece_id=PPVKNKAJSKEX5I3R6YHXLLX6G3CUULQKUI3E2GCHIRK6JGN75XUQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.053220935 (68h56m6.484301066s) [noise=true , address=104.36.82.46:4499 , piece_id=YRHJTQXPQXZWDF374EOEQ64PNIKAGHC5X7DJAYXGK46J42SEYRBA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:13.262805252 (68h56m7.693885383s) [upload=71849]: Next returned nil segment Aug 11 14:22:13.262823657 (68h56m7.693903788s) [upload=71849]: Waiting for error group managing segments... Aug 11 14:22:13.588627970 (68h56m8.019708101s) [noise=true , address=64.49.13.154:4933 , piece_id=GYGTGPN2MG5ETRMPZSVWE54O5F56GGWULDZJFZFESE4LTZWMSPQQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:13.648017643 (68h56m8.079097774s) [noise=true , address=64.49.13.170:4721 , piece_id=KDZC5YVY3TRAUWYDHJAVQWMANLPNFKFAO4SCUDK5XVVHUEEBY5WQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.011466053 (68h56m8.442546184s) [noise=true , address=104.36.82.17:4339 , piece_id=5YHVPEINCZVUQU75Z3PON57QRECP6LADNQZTHPQMLE7ZFH24AKNQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.011959307 (68h56m8.443039438s) [noise=true , address=77.90.31.13:4643 , piece_id=7ITQU24YQWV5474J5K3ZZNKAEGRQ6FHCYB4EW2BCDPKDAIXKIP6A, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.047939761 (68h56m8.479019892s) [noise=true , address=104.36.82.18:4380 , piece_id=QD2IEZ7YMUHDA7P7E5THL6LDKS7KJTXWBVSJUUUFZY5H4SZZCTAQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.056207570 (68h56m8.487287691s) [noise=true , address=104.36.82.8:4999 , piece_id=CXT4YR5NWU4NBIXPBHNIRTVOW5LZYFADEVVLGFIGQXYWPS772M7Q, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.107257497 (68h56m8.538337628s) [noise=true , address=104.36.82.35:4594 , piece_id=K2WAYZOFR5IWKKGIGV6QM6C62E47RR7NGCX47ZP4H4QFTOXLTW2Q, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.333173827 (68h56m8.764253948s) [noise=true , address=104.36.82.21:4489 , piece_id=URWEO4323RZN6YV6UOCK4PC3UXIKMQIEOZLCL24FQ33DXKCIKOOQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.374631602 (68h56m8.805711733s) [noise=true , address=104.36.82.22:4537 , piece_id=SIWG45D42XCBWWLVCUUQQH3X2AINV3RAJYJ7J3NRICU7A5PLIO6A, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.384903195 (68h56m8.815983326s) [noise=true , address=104.36.82.62:4417 , piece_id=NNEGV2OM54WCS4S7XSXRXLSK4IJHYGHLT26TVNUZQDA67JWDTAAA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.497638849 (68h56m8.92871897s) [noise=true , address=104.36.82.56:4255 , piece_id=XTYNJXNDMWUXIXALMID4QZ3KTC4RYHOHUUEHC3GRE23PWXFO2QXQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.575860717 (68h56m9.006940848s) [noise=true , address=104.36.82.47:4549 , piece_id=MJQPFWASTOO2EYBJKKQQQUEZBZIE5ZUIUEQUMM5QLPR2LJTIXLRA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.911894362 (68h56m9.342974523s) [noise=true , address=104.36.82.37:4674 , piece_id=5IUTA2L4TCVI73IPYPSTDENQNFX2N4ZSF7EMEBQRAWDL2ARKSUPQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:15.160712381 (68h56m9.591792502s) [noise=true , address=104.36.82.44:4871 , piece_id=ASZ6X7K4BHSYQID47L6AFWZWA5WC43MRMK4CZPUYQLL4EDRAU6WQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:15.160743531 (68h56m9.591823662s) [seg_pos={0 0}, upload=71849]: Segment reached optimal threshold of 14 pieces. Aug 11 14:22:15.160870043 (68h56m9.591950174s) [noise=true , address=104.36.82.46:4499 , piece_id=YRHJTQXPQXZWDF374EOEQ64PNIKAGHC5X7DJAYXGK46J42SEYRBA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.160916933 (68h56m9.591997064s) [noise=true , address=104.36.82.60:4360 , piece_id=CSSK655RYUWKXZP646LKO3BBPHSGBWPSB2QMAI4LKHR7SPC2PGRQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T24rtYBJ47E6Ef1ZZKMWcpBrWBpCg5QV2ZHniCZZkzJF2yUNk): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.161113549 (68h56m9.59219368s) [noise=true , address=104.36.82.39:4739 , piece_id=B5QJU4VPBYUK5247SR6SCNTBU42Q3CBDK4Z3HGHNLAL76MGAENHA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection; write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection Aug 11 14:22:15.161177131 (68h56m9.592257282s) [noise=true , address=104.36.82.58:4272 , piece_id=PPVKNKAJSKEX5I3R6YHXLLX6G3CUULQKUI3E2GCHIRK6JGN75XUQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yY39UpswsTLoWz4353JykK35fg8KB4AJWPatKGFgqTt4gDMcR): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.161247746 (68h56m9.592327897s) [seg_pos={0 0}, upload=71849]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:22:15.161376632 (68h56m9.592456783s) [upload=71849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:22:15.161384037 (68h56m9.592464188s) [upload=71849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:15.481549940 (68h56m9.912630061s) [upload=71849]: Done uploading segments... Aug 11 14:22:15.481569789 (68h56m9.91264992s) [upload=71849]: Upload finished. err: <nil> Duration: 7.512693479s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 9.231825013s FAIL (2025-08-11 14:24:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.231847516s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X..................X..............................X.X..] duration: 6.685673189s FAIL (2025-08-11 14:23:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 14:22:11.588492069 (68h56m6.0195722s) [upload=71849]: Starting upload Aug 11 14:22:11.588534811 (68h56m6.019614932s) [upload=71849]: Done starting upload Aug 11 14:22:11.588641926 (68h56m6.019722047s) [upload=71849]: Uploading segments... Aug 11 14:22:11.588701349 (68h56m6.01978148s) [upload=71849]: Got next segment. Inline: false Aug 11 14:22:11.588729163 (68h56m6.019809294s) [upload=71849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:11.588733110 (68h56m6.019813241s) [upload=71849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:22:12.047436437 (68h56m6.478516568s) [seg_pos={0 0}, upload=71849]: Begin upload segment... Aug 11 14:22:12.047509667 (68h56m6.478589828s) [seg_pos={0 0}, upload=71849]: Done begin upload segment. Aug 11 14:22:12.047525597 (68h56m6.478605728s) [noise=true , address=104.36.82.60:4360 , piece_id=CSSK655RYUWKXZP646LKO3BBPHSGBWPSB2QMAI4LKHR7SPC2PGRQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.047541838 (68h56m6.478621969s) [noise=true , address=104.36.82.47:4549 , piece_id=MJQPFWASTOO2EYBJKKQQQUEZBZIE5ZUIUEQUMM5QLPR2LJTIXLRA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.048025404 (68h56m6.479105535s) [noise=true , address=64.49.13.154:4933 , piece_id=GYGTGPN2MG5ETRMPZSVWE54O5F56GGWULDZJFZFESE4LTZWMSPQQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.048991011 (68h56m6.480071142s) [noise=true , address=104.36.82.21:4489 , piece_id=URWEO4323RZN6YV6UOCK4PC3UXIKMQIEOZLCL24FQ33DXKCIKOOQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.049350639 (68h56m6.48043078s) [noise=true , address=104.36.82.17:4339 , piece_id=5YHVPEINCZVUQU75Z3PON57QRECP6LADNQZTHPQMLE7ZFH24AKNQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.049695530 (68h56m6.480775661s) [noise=true , address=77.90.31.13:4643 , piece_id=7ITQU24YQWV5474J5K3ZZNKAEGRQ6FHCYB4EW2BCDPKDAIXKIP6A, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050032955 (68h56m6.481113096s) [noise=true , address=104.36.82.8:4999 , piece_id=CXT4YR5NWU4NBIXPBHNIRTVOW5LZYFADEVVLGFIGQXYWPS772M7Q, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050304054 (68h56m6.481384175s) [noise=true , address=104.36.82.37:4674 , piece_id=5IUTA2L4TCVI73IPYPSTDENQNFX2N4ZSF7EMEBQRAWDL2ARKSUPQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050622062 (68h56m6.481702183s) [noise=true , address=104.36.82.39:4739 , piece_id=B5QJU4VPBYUK5247SR6SCNTBU42Q3CBDK4Z3HGHNLAL76MGAENHA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050944299 (68h56m6.48202443s) [noise=true , address=104.36.82.22:4537 , piece_id=SIWG45D42XCBWWLVCUUQQH3X2AINV3RAJYJ7J3NRICU7A5PLIO6A, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051263940 (68h56m6.482344072s) [noise=true , address=104.36.82.44:4871 , piece_id=ASZ6X7K4BHSYQID47L6AFWZWA5WC43MRMK4CZPUYQLL4EDRAU6WQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051533416 (68h56m6.482613547s) [noise=true , address=104.36.82.35:4594 , piece_id=K2WAYZOFR5IWKKGIGV6QM6C62E47RR7NGCX47ZP4H4QFTOXLTW2Q, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051831166 (68h56m6.482911297s) [noise=true , address=104.36.82.56:4255 , piece_id=XTYNJXNDMWUXIXALMID4QZ3KTC4RYHOHUUEHC3GRE23PWXFO2QXQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052168281 (68h56m6.483248402s) [noise=true , address=64.49.13.170:4721 , piece_id=KDZC5YVY3TRAUWYDHJAVQWMANLPNFKFAO4SCUDK5XVVHUEEBY5WQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052226993 (68h56m6.483307124s) [noise=true , address=104.36.82.62:4417 , piece_id=NNEGV2OM54WCS4S7XSXRXLSK4IJHYGHLT26TVNUZQDA67JWDTAAA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052613883 (68h56m6.483694014s) [noise=true , address=104.36.82.18:4380 , piece_id=QD2IEZ7YMUHDA7P7E5THL6LDKS7KJTXWBVSJUUUFZY5H4SZZCTAQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052921051 (68h56m6.484001182s) [noise=true , address=104.36.82.58:4272 , piece_id=PPVKNKAJSKEX5I3R6YHXLLX6G3CUULQKUI3E2GCHIRK6JGN75XUQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.053220935 (68h56m6.484301066s) [noise=true , address=104.36.82.46:4499 , piece_id=YRHJTQXPQXZWDF374EOEQ64PNIKAGHC5X7DJAYXGK46J42SEYRBA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:13.262805252 (68h56m7.693885383s) [upload=71849]: Next returned nil segment Aug 11 14:22:13.262823657 (68h56m7.693903788s) [upload=71849]: Waiting for error group managing segments... Aug 11 14:22:13.588627970 (68h56m8.019708101s) [noise=true , address=64.49.13.154:4933 , piece_id=GYGTGPN2MG5ETRMPZSVWE54O5F56GGWULDZJFZFESE4LTZWMSPQQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:13.648017643 (68h56m8.079097774s) [noise=true , address=64.49.13.170:4721 , piece_id=KDZC5YVY3TRAUWYDHJAVQWMANLPNFKFAO4SCUDK5XVVHUEEBY5WQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.011466053 (68h56m8.442546184s) [noise=true , address=104.36.82.17:4339 , piece_id=5YHVPEINCZVUQU75Z3PON57QRECP6LADNQZTHPQMLE7ZFH24AKNQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.011959307 (68h56m8.443039438s) [noise=true , address=77.90.31.13:4643 , piece_id=7ITQU24YQWV5474J5K3ZZNKAEGRQ6FHCYB4EW2BCDPKDAIXKIP6A, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.047939761 (68h56m8.479019892s) [noise=true , address=104.36.82.18:4380 , piece_id=QD2IEZ7YMUHDA7P7E5THL6LDKS7KJTXWBVSJUUUFZY5H4SZZCTAQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.056207570 (68h56m8.487287691s) [noise=true , address=104.36.82.8:4999 , piece_id=CXT4YR5NWU4NBIXPBHNIRTVOW5LZYFADEVVLGFIGQXYWPS772M7Q, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.107257497 (68h56m8.538337628s) [noise=true , address=104.36.82.35:4594 , piece_id=K2WAYZOFR5IWKKGIGV6QM6C62E47RR7NGCX47ZP4H4QFTOXLTW2Q, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.333173827 (68h56m8.764253948s) [noise=true , address=104.36.82.21:4489 , piece_id=URWEO4323RZN6YV6UOCK4PC3UXIKMQIEOZLCL24FQ33DXKCIKOOQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.374631602 (68h56m8.805711733s) [noise=true , address=104.36.82.22:4537 , piece_id=SIWG45D42XCBWWLVCUUQQH3X2AINV3RAJYJ7J3NRICU7A5PLIO6A, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.384903195 (68h56m8.815983326s) [noise=true , address=104.36.82.62:4417 , piece_id=NNEGV2OM54WCS4S7XSXRXLSK4IJHYGHLT26TVNUZQDA67JWDTAAA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.497638849 (68h56m8.92871897s) [noise=true , address=104.36.82.56:4255 , piece_id=XTYNJXNDMWUXIXALMID4QZ3KTC4RYHOHUUEHC3GRE23PWXFO2QXQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.575860717 (68h56m9.006940848s) [noise=true , address=104.36.82.47:4549 , piece_id=MJQPFWASTOO2EYBJKKQQQUEZBZIE5ZUIUEQUMM5QLPR2LJTIXLRA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.911894362 (68h56m9.342974523s) [noise=true , address=104.36.82.37:4674 , piece_id=5IUTA2L4TCVI73IPYPSTDENQNFX2N4ZSF7EMEBQRAWDL2ARKSUPQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:15.160712381 (68h56m9.591792502s) [noise=true , address=104.36.82.44:4871 , piece_id=ASZ6X7K4BHSYQID47L6AFWZWA5WC43MRMK4CZPUYQLL4EDRAU6WQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:15.160743531 (68h56m9.591823662s) [seg_pos={0 0}, upload=71849]: Segment reached optimal threshold of 14 pieces. Aug 11 14:22:15.160870043 (68h56m9.591950174s) [noise=true , address=104.36.82.46:4499 , piece_id=YRHJTQXPQXZWDF374EOEQ64PNIKAGHC5X7DJAYXGK46J42SEYRBA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.160916933 (68h56m9.591997064s) [noise=true , address=104.36.82.60:4360 , piece_id=CSSK655RYUWKXZP646LKO3BBPHSGBWPSB2QMAI4LKHR7SPC2PGRQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T24rtYBJ47E6Ef1ZZKMWcpBrWBpCg5QV2ZHniCZZkzJF2yUNk): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.161113549 (68h56m9.59219368s) [noise=true , address=104.36.82.39:4739 , piece_id=B5QJU4VPBYUK5247SR6SCNTBU42Q3CBDK4Z3HGHNLAL76MGAENHA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection; write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection Aug 11 14:22:15.161177131 (68h56m9.592257282s) [noise=true , address=104.36.82.58:4272 , piece_id=PPVKNKAJSKEX5I3R6YHXLLX6G3CUULQKUI3E2GCHIRK6JGN75XUQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yY39UpswsTLoWz4353JykK35fg8KB4AJWPatKGFgqTt4gDMcR): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.161247746 (68h56m9.592327897s) [seg_pos={0 0}, upload=71849]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:22:15.161376632 (68h56m9.592456783s) [upload=71849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:22:15.161384037 (68h56m9.592464188s) [upload=71849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:15.481549940 (68h56m9.912630061s) [upload=71849]: Done uploading segments... Aug 11 14:22:15.481569789 (68h56m9.91264992s) [upload=71849]: Upload finished. err: <nil> Duration: 6.685699369s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [....X................X......X................X............X.] duration: 1.865906899s FAIL (2025-08-11 14:23:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 1.865934532s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 9.900598028s FAIL (2025-08-11 14:23:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.900623157s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X..................X..............................X.X.] duration: 7.662152106s FAIL (2025-08-11 14:22:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 14:22:11.588492069 (68h56m6.0195722s) [upload=71849]: Starting upload Aug 11 14:22:11.588534811 (68h56m6.019614932s) [upload=71849]: Done starting upload Aug 11 14:22:11.588641926 (68h56m6.019722047s) [upload=71849]: Uploading segments... Aug 11 14:22:11.588701349 (68h56m6.01978148s) [upload=71849]: Got next segment. Inline: false Aug 11 14:22:11.588729163 (68h56m6.019809294s) [upload=71849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:11.588733110 (68h56m6.019813241s) [upload=71849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:22:12.047436437 (68h56m6.478516568s) [seg_pos={0 0}, upload=71849]: Begin upload segment... Aug 11 14:22:12.047509667 (68h56m6.478589828s) [seg_pos={0 0}, upload=71849]: Done begin upload segment. Aug 11 14:22:12.047525597 (68h56m6.478605728s) [noise=true , address=104.36.82.60:4360 , piece_id=CSSK655RYUWKXZP646LKO3BBPHSGBWPSB2QMAI4LKHR7SPC2PGRQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.047541838 (68h56m6.478621969s) [noise=true , address=104.36.82.47:4549 , piece_id=MJQPFWASTOO2EYBJKKQQQUEZBZIE5ZUIUEQUMM5QLPR2LJTIXLRA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.048025404 (68h56m6.479105535s) [noise=true , address=64.49.13.154:4933 , piece_id=GYGTGPN2MG5ETRMPZSVWE54O5F56GGWULDZJFZFESE4LTZWMSPQQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.048991011 (68h56m6.480071142s) [noise=true , address=104.36.82.21:4489 , piece_id=URWEO4323RZN6YV6UOCK4PC3UXIKMQIEOZLCL24FQ33DXKCIKOOQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.049350639 (68h56m6.48043078s) [noise=true , address=104.36.82.17:4339 , piece_id=5YHVPEINCZVUQU75Z3PON57QRECP6LADNQZTHPQMLE7ZFH24AKNQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.049695530 (68h56m6.480775661s) [noise=true , address=77.90.31.13:4643 , piece_id=7ITQU24YQWV5474J5K3ZZNKAEGRQ6FHCYB4EW2BCDPKDAIXKIP6A, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050032955 (68h56m6.481113096s) [noise=true , address=104.36.82.8:4999 , piece_id=CXT4YR5NWU4NBIXPBHNIRTVOW5LZYFADEVVLGFIGQXYWPS772M7Q, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050304054 (68h56m6.481384175s) [noise=true , address=104.36.82.37:4674 , piece_id=5IUTA2L4TCVI73IPYPSTDENQNFX2N4ZSF7EMEBQRAWDL2ARKSUPQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050622062 (68h56m6.481702183s) [noise=true , address=104.36.82.39:4739 , piece_id=B5QJU4VPBYUK5247SR6SCNTBU42Q3CBDK4Z3HGHNLAL76MGAENHA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050944299 (68h56m6.48202443s) [noise=true , address=104.36.82.22:4537 , piece_id=SIWG45D42XCBWWLVCUUQQH3X2AINV3RAJYJ7J3NRICU7A5PLIO6A, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051263940 (68h56m6.482344072s) [noise=true , address=104.36.82.44:4871 , piece_id=ASZ6X7K4BHSYQID47L6AFWZWA5WC43MRMK4CZPUYQLL4EDRAU6WQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051533416 (68h56m6.482613547s) [noise=true , address=104.36.82.35:4594 , piece_id=K2WAYZOFR5IWKKGIGV6QM6C62E47RR7NGCX47ZP4H4QFTOXLTW2Q, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051831166 (68h56m6.482911297s) [noise=true , address=104.36.82.56:4255 , piece_id=XTYNJXNDMWUXIXALMID4QZ3KTC4RYHOHUUEHC3GRE23PWXFO2QXQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052168281 (68h56m6.483248402s) [noise=true , address=64.49.13.170:4721 , piece_id=KDZC5YVY3TRAUWYDHJAVQWMANLPNFKFAO4SCUDK5XVVHUEEBY5WQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052226993 (68h56m6.483307124s) [noise=true , address=104.36.82.62:4417 , piece_id=NNEGV2OM54WCS4S7XSXRXLSK4IJHYGHLT26TVNUZQDA67JWDTAAA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052613883 (68h56m6.483694014s) [noise=true , address=104.36.82.18:4380 , piece_id=QD2IEZ7YMUHDA7P7E5THL6LDKS7KJTXWBVSJUUUFZY5H4SZZCTAQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052921051 (68h56m6.484001182s) [noise=true , address=104.36.82.58:4272 , piece_id=PPVKNKAJSKEX5I3R6YHXLLX6G3CUULQKUI3E2GCHIRK6JGN75XUQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.053220935 (68h56m6.484301066s) [noise=true , address=104.36.82.46:4499 , piece_id=YRHJTQXPQXZWDF374EOEQ64PNIKAGHC5X7DJAYXGK46J42SEYRBA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:13.262805252 (68h56m7.693885383s) [upload=71849]: Next returned nil segment Aug 11 14:22:13.262823657 (68h56m7.693903788s) [upload=71849]: Waiting for error group managing segments... Aug 11 14:22:13.588627970 (68h56m8.019708101s) [noise=true , address=64.49.13.154:4933 , piece_id=GYGTGPN2MG5ETRMPZSVWE54O5F56GGWULDZJFZFESE4LTZWMSPQQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:13.648017643 (68h56m8.079097774s) [noise=true , address=64.49.13.170:4721 , piece_id=KDZC5YVY3TRAUWYDHJAVQWMANLPNFKFAO4SCUDK5XVVHUEEBY5WQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.011466053 (68h56m8.442546184s) [noise=true , address=104.36.82.17:4339 , piece_id=5YHVPEINCZVUQU75Z3PON57QRECP6LADNQZTHPQMLE7ZFH24AKNQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.011959307 (68h56m8.443039438s) [noise=true , address=77.90.31.13:4643 , piece_id=7ITQU24YQWV5474J5K3ZZNKAEGRQ6FHCYB4EW2BCDPKDAIXKIP6A, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.047939761 (68h56m8.479019892s) [noise=true , address=104.36.82.18:4380 , piece_id=QD2IEZ7YMUHDA7P7E5THL6LDKS7KJTXWBVSJUUUFZY5H4SZZCTAQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.056207570 (68h56m8.487287691s) [noise=true , address=104.36.82.8:4999 , piece_id=CXT4YR5NWU4NBIXPBHNIRTVOW5LZYFADEVVLGFIGQXYWPS772M7Q, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.107257497 (68h56m8.538337628s) [noise=true , address=104.36.82.35:4594 , piece_id=K2WAYZOFR5IWKKGIGV6QM6C62E47RR7NGCX47ZP4H4QFTOXLTW2Q, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.333173827 (68h56m8.764253948s) [noise=true , address=104.36.82.21:4489 , piece_id=URWEO4323RZN6YV6UOCK4PC3UXIKMQIEOZLCL24FQ33DXKCIKOOQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.374631602 (68h56m8.805711733s) [noise=true , address=104.36.82.22:4537 , piece_id=SIWG45D42XCBWWLVCUUQQH3X2AINV3RAJYJ7J3NRICU7A5PLIO6A, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.384903195 (68h56m8.815983326s) [noise=true , address=104.36.82.62:4417 , piece_id=NNEGV2OM54WCS4S7XSXRXLSK4IJHYGHLT26TVNUZQDA67JWDTAAA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.497638849 (68h56m8.92871897s) [noise=true , address=104.36.82.56:4255 , piece_id=XTYNJXNDMWUXIXALMID4QZ3KTC4RYHOHUUEHC3GRE23PWXFO2QXQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.575860717 (68h56m9.006940848s) [noise=true , address=104.36.82.47:4549 , piece_id=MJQPFWASTOO2EYBJKKQQQUEZBZIE5ZUIUEQUMM5QLPR2LJTIXLRA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.911894362 (68h56m9.342974523s) [noise=true , address=104.36.82.37:4674 , piece_id=5IUTA2L4TCVI73IPYPSTDENQNFX2N4ZSF7EMEBQRAWDL2ARKSUPQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:15.160712381 (68h56m9.591792502s) [noise=true , address=104.36.82.44:4871 , piece_id=ASZ6X7K4BHSYQID47L6AFWZWA5WC43MRMK4CZPUYQLL4EDRAU6WQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:15.160743531 (68h56m9.591823662s) [seg_pos={0 0}, upload=71849]: Segment reached optimal threshold of 14 pieces. Aug 11 14:22:15.160870043 (68h56m9.591950174s) [noise=true , address=104.36.82.46:4499 , piece_id=YRHJTQXPQXZWDF374EOEQ64PNIKAGHC5X7DJAYXGK46J42SEYRBA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.160916933 (68h56m9.591997064s) [noise=true , address=104.36.82.60:4360 , piece_id=CSSK655RYUWKXZP646LKO3BBPHSGBWPSB2QMAI4LKHR7SPC2PGRQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T24rtYBJ47E6Ef1ZZKMWcpBrWBpCg5QV2ZHniCZZkzJF2yUNk): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.161113549 (68h56m9.59219368s) [noise=true , address=104.36.82.39:4739 , piece_id=B5QJU4VPBYUK5247SR6SCNTBU42Q3CBDK4Z3HGHNLAL76MGAENHA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection; write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection Aug 11 14:22:15.161177131 (68h56m9.592257282s) [noise=true , address=104.36.82.58:4272 , piece_id=PPVKNKAJSKEX5I3R6YHXLLX6G3CUULQKUI3E2GCHIRK6JGN75XUQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yY39UpswsTLoWz4353JykK35fg8KB4AJWPatKGFgqTt4gDMcR): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.161247746 (68h56m9.592327897s) [seg_pos={0 0}, upload=71849]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:22:15.161376632 (68h56m9.592456783s) [upload=71849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:22:15.161384037 (68h56m9.592464188s) [upload=71849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:15.481549940 (68h56m9.912630061s) [upload=71849]: Done uploading segments... Aug 11 14:22:15.481569789 (68h56m9.91264992s) [upload=71849]: Upload finished. err: <nil> Duration: 7.662189528s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.034231894s FAIL (2025-08-11 14:22:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.034252973s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X......X..................X..............................X.X] duration: 5.219959931s FAIL (2025-08-11 14:22:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 14:22:11.588492069 (68h56m6.0195722s) [upload=71849]: Starting upload Aug 11 14:22:11.588534811 (68h56m6.019614932s) [upload=71849]: Done starting upload Aug 11 14:22:11.588641926 (68h56m6.019722047s) [upload=71849]: Uploading segments... Aug 11 14:22:11.588701349 (68h56m6.01978148s) [upload=71849]: Got next segment. Inline: false Aug 11 14:22:11.588729163 (68h56m6.019809294s) [upload=71849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:11.588733110 (68h56m6.019813241s) [upload=71849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:22:12.047436437 (68h56m6.478516568s) [seg_pos={0 0}, upload=71849]: Begin upload segment... Aug 11 14:22:12.047509667 (68h56m6.478589828s) [seg_pos={0 0}, upload=71849]: Done begin upload segment. Aug 11 14:22:12.047525597 (68h56m6.478605728s) [noise=true , address=104.36.82.60:4360 , piece_id=CSSK655RYUWKXZP646LKO3BBPHSGBWPSB2QMAI4LKHR7SPC2PGRQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.047541838 (68h56m6.478621969s) [noise=true , address=104.36.82.47:4549 , piece_id=MJQPFWASTOO2EYBJKKQQQUEZBZIE5ZUIUEQUMM5QLPR2LJTIXLRA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.048025404 (68h56m6.479105535s) [noise=true , address=64.49.13.154:4933 , piece_id=GYGTGPN2MG5ETRMPZSVWE54O5F56GGWULDZJFZFESE4LTZWMSPQQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.048991011 (68h56m6.480071142s) [noise=true , address=104.36.82.21:4489 , piece_id=URWEO4323RZN6YV6UOCK4PC3UXIKMQIEOZLCL24FQ33DXKCIKOOQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.049350639 (68h56m6.48043078s) [noise=true , address=104.36.82.17:4339 , piece_id=5YHVPEINCZVUQU75Z3PON57QRECP6LADNQZTHPQMLE7ZFH24AKNQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.049695530 (68h56m6.480775661s) [noise=true , address=77.90.31.13:4643 , piece_id=7ITQU24YQWV5474J5K3ZZNKAEGRQ6FHCYB4EW2BCDPKDAIXKIP6A, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050032955 (68h56m6.481113096s) [noise=true , address=104.36.82.8:4999 , piece_id=CXT4YR5NWU4NBIXPBHNIRTVOW5LZYFADEVVLGFIGQXYWPS772M7Q, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050304054 (68h56m6.481384175s) [noise=true , address=104.36.82.37:4674 , piece_id=5IUTA2L4TCVI73IPYPSTDENQNFX2N4ZSF7EMEBQRAWDL2ARKSUPQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050622062 (68h56m6.481702183s) [noise=true , address=104.36.82.39:4739 , piece_id=B5QJU4VPBYUK5247SR6SCNTBU42Q3CBDK4Z3HGHNLAL76MGAENHA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.050944299 (68h56m6.48202443s) [noise=true , address=104.36.82.22:4537 , piece_id=SIWG45D42XCBWWLVCUUQQH3X2AINV3RAJYJ7J3NRICU7A5PLIO6A, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051263940 (68h56m6.482344072s) [noise=true , address=104.36.82.44:4871 , piece_id=ASZ6X7K4BHSYQID47L6AFWZWA5WC43MRMK4CZPUYQLL4EDRAU6WQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051533416 (68h56m6.482613547s) [noise=true , address=104.36.82.35:4594 , piece_id=K2WAYZOFR5IWKKGIGV6QM6C62E47RR7NGCX47ZP4H4QFTOXLTW2Q, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.051831166 (68h56m6.482911297s) [noise=true , address=104.36.82.56:4255 , piece_id=XTYNJXNDMWUXIXALMID4QZ3KTC4RYHOHUUEHC3GRE23PWXFO2QXQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052168281 (68h56m6.483248402s) [noise=true , address=64.49.13.170:4721 , piece_id=KDZC5YVY3TRAUWYDHJAVQWMANLPNFKFAO4SCUDK5XVVHUEEBY5WQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052226993 (68h56m6.483307124s) [noise=true , address=104.36.82.62:4417 , piece_id=NNEGV2OM54WCS4S7XSXRXLSK4IJHYGHLT26TVNUZQDA67JWDTAAA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052613883 (68h56m6.483694014s) [noise=true , address=104.36.82.18:4380 , piece_id=QD2IEZ7YMUHDA7P7E5THL6LDKS7KJTXWBVSJUUUFZY5H4SZZCTAQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.052921051 (68h56m6.484001182s) [noise=true , address=104.36.82.58:4272 , piece_id=PPVKNKAJSKEX5I3R6YHXLLX6G3CUULQKUI3E2GCHIRK6JGN75XUQ, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:12.053220935 (68h56m6.484301066s) [noise=true , address=104.36.82.46:4499 , piece_id=YRHJTQXPQXZWDF374EOEQ64PNIKAGHC5X7DJAYXGK46J42SEYRBA, seg_pos={0 0}, upload=71849]: Uploading piece... Aug 11 14:22:13.262805252 (68h56m7.693885383s) [upload=71849]: Next returned nil segment Aug 11 14:22:13.262823657 (68h56m7.693903788s) [upload=71849]: Waiting for error group managing segments... Aug 11 14:22:13.588627970 (68h56m8.019708101s) [noise=true , address=64.49.13.154:4933 , piece_id=GYGTGPN2MG5ETRMPZSVWE54O5F56GGWULDZJFZFESE4LTZWMSPQQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:13.648017643 (68h56m8.079097774s) [noise=true , address=64.49.13.170:4721 , piece_id=KDZC5YVY3TRAUWYDHJAVQWMANLPNFKFAO4SCUDK5XVVHUEEBY5WQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.011466053 (68h56m8.442546184s) [noise=true , address=104.36.82.17:4339 , piece_id=5YHVPEINCZVUQU75Z3PON57QRECP6LADNQZTHPQMLE7ZFH24AKNQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.011959307 (68h56m8.443039438s) [noise=true , address=77.90.31.13:4643 , piece_id=7ITQU24YQWV5474J5K3ZZNKAEGRQ6FHCYB4EW2BCDPKDAIXKIP6A, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.047939761 (68h56m8.479019892s) [noise=true , address=104.36.82.18:4380 , piece_id=QD2IEZ7YMUHDA7P7E5THL6LDKS7KJTXWBVSJUUUFZY5H4SZZCTAQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.056207570 (68h56m8.487287691s) [noise=true , address=104.36.82.8:4999 , piece_id=CXT4YR5NWU4NBIXPBHNIRTVOW5LZYFADEVVLGFIGQXYWPS772M7Q, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.107257497 (68h56m8.538337628s) [noise=true , address=104.36.82.35:4594 , piece_id=K2WAYZOFR5IWKKGIGV6QM6C62E47RR7NGCX47ZP4H4QFTOXLTW2Q, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.333173827 (68h56m8.764253948s) [noise=true , address=104.36.82.21:4489 , piece_id=URWEO4323RZN6YV6UOCK4PC3UXIKMQIEOZLCL24FQ33DXKCIKOOQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.374631602 (68h56m8.805711733s) [noise=true , address=104.36.82.22:4537 , piece_id=SIWG45D42XCBWWLVCUUQQH3X2AINV3RAJYJ7J3NRICU7A5PLIO6A, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.384903195 (68h56m8.815983326s) [noise=true , address=104.36.82.62:4417 , piece_id=NNEGV2OM54WCS4S7XSXRXLSK4IJHYGHLT26TVNUZQDA67JWDTAAA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.497638849 (68h56m8.92871897s) [noise=true , address=104.36.82.56:4255 , piece_id=XTYNJXNDMWUXIXALMID4QZ3KTC4RYHOHUUEHC3GRE23PWXFO2QXQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.575860717 (68h56m9.006940848s) [noise=true , address=104.36.82.47:4549 , piece_id=MJQPFWASTOO2EYBJKKQQQUEZBZIE5ZUIUEQUMM5QLPR2LJTIXLRA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:14.911894362 (68h56m9.342974523s) [noise=true , address=104.36.82.37:4674 , piece_id=5IUTA2L4TCVI73IPYPSTDENQNFX2N4ZSF7EMEBQRAWDL2ARKSUPQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:15.160712381 (68h56m9.591792502s) [noise=true , address=104.36.82.44:4871 , piece_id=ASZ6X7K4BHSYQID47L6AFWZWA5WC43MRMK4CZPUYQLL4EDRAU6WQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: <nil> Aug 11 14:22:15.160743531 (68h56m9.591823662s) [seg_pos={0 0}, upload=71849]: Segment reached optimal threshold of 14 pieces. Aug 11 14:22:15.160870043 (68h56m9.591950174s) [noise=true , address=104.36.82.46:4499 , piece_id=YRHJTQXPQXZWDF374EOEQ64PNIKAGHC5X7DJAYXGK46J42SEYRBA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.160916933 (68h56m9.591997064s) [noise=true , address=104.36.82.60:4360 , piece_id=CSSK655RYUWKXZP646LKO3BBPHSGBWPSB2QMAI4LKHR7SPC2PGRQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T24rtYBJ47E6Ef1ZZKMWcpBrWBpCg5QV2ZHniCZZkzJF2yUNk): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.161113549 (68h56m9.59219368s) [noise=true , address=104.36.82.39:4739 , piece_id=B5QJU4VPBYUK5247SR6SCNTBU42Q3CBDK4Z3HGHNLAL76MGAENHA, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection; write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60416->104.36.82.39:4739: use of closed network connection Aug 11 14:22:15.161177131 (68h56m9.592257282s) [noise=true , address=104.36.82.58:4272 , piece_id=PPVKNKAJSKEX5I3R6YHXLLX6G3CUULQKUI3E2GCHIRK6JGN75XUQ, seg_pos={0 0}, upload=71849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yY39UpswsTLoWz4353JykK35fg8KB4AJWPatKGFgqTt4gDMcR): protocol: expected piece hash; context canceled; EOF Aug 11 14:22:15.161247746 (68h56m9.592327897s) [seg_pos={0 0}, upload=71849]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:22:15.161376632 (68h56m9.592456783s) [upload=71849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:22:15.161384037 (68h56m9.592464188s) [upload=71849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:15.481549940 (68h56m9.912630061s) [upload=71849]: Done uploading segments... Aug 11 14:22:15.481569789 (68h56m9.91264992s) [upload=71849]: Upload finished. err: <nil> Duration: 5.21999046s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.....X................X......X................X............X] duration: 1.612111649s FAIL (2025-08-11 14:22:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 11 14:22:09.180495965 (68h56m3.611576126s) [upload=71848]: Starting upload Aug 11 14:22:09.180591568 (68h56m3.611671699s) [upload=71848]: Done starting upload Aug 11 14:22:09.180690927 (68h56m3.611771079s) [upload=71848]: Uploading segments... Aug 11 14:22:09.180752797 (68h56m3.611832918s) [upload=71848]: Got next segment. Inline: true Aug 11 14:22:09.180757686 (68h56m3.611837817s) [upload=71848]: Waiting for error group managing segments... Aug 11 14:22:09.180792712 (68h56m3.611872844s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:22:09.180797021 (68h56m3.611877152s) [upload=71848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:22:09.180799375 (68h56m3.611879506s) [upload=71848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:22:09.477891571 (68h56m3.908971722s) [upload=71848]: Done uploading segments... Aug 11 14:22:09.477914635 (68h56m3.908994766s) [upload=71848]: Upload finished. err: <nil> Duration:* 1.612138019s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.224195565s FAIL (2025-08-11 14:21:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.224262854s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X......X..................X..............................X.] duration: 7.083720713s FAIL (2025-08-11 14:21:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 14:20:25.453217472 (68h54m19.884297604s) [upload=71821]: Starting upload Aug 11 14:20:25.453256097 (68h54m19.884336228s) [upload=71821]: Done starting upload Aug 11 14:20:25.453429559 (68h54m19.88450969s) [upload=71821]: Uploading segments... Aug 11 14:20:25.453462020 (68h54m19.884542152s) [upload=71821]: Got next segment. Inline: false Aug 11 14:20:25.453484393 (68h54m19.884564525s) [upload=71821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:20:25.453488852 (68h54m19.884568973s) [upload=71821]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:20:25.904819974 (68h54m20.335900105s) [seg_pos={0 0}, upload=71821]: Begin upload segment... Aug 11 14:20:25.904919054 (68h54m20.335999185s) [seg_pos={0 0}, upload=71821]: Done begin upload segment. Aug 11 14:20:25.904942519 (68h54m20.33602265s) [noise=true , address=104.36.82.59:4307 , piece_id=WMLIRN56JTUYCVIFLK77B6T7LMWKOWJZEFEX4VMCKLPHX4PERMGQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.904966685 (68h54m20.336046806s) [noise=true , address=77.90.31.13:4639 , piece_id=2TR2AB4NLZTCIUDAFIZEYR5W3BUBSVPAMSOKNX32HQ6527KDXG7Q, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.905322626 (68h54m20.336402747s) [noise=true , address=104.36.82.35:4605 , piece_id=77ER5KLFOPIVKQWZMTYQV3GAA7ASMKQDD5AXNCZ3HTZOR36O2KXQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.905450511 (68h54m20.336530632s) [noise=true , address=104.36.82.5:4851 , piece_id=CNDTEW4A5HF3ZV44ZSS3GIXDOB6SEI7HFCB3OVZJLSBILCKEFSMA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.905619344 (68h54m20.336699475s) [noise=true , address=104.36.82.23:4451 , piece_id=3RWYSPA5GZY5BXSAVP45WMHNTWL6KLL5MPK2QFCZX6MMYUDOX3SQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.905963603 (68h54m20.337043734s) [noise=true , address=104.36.82.45:4920 , piece_id=BKPN6F3FSG7Y2WZS5EUYUTS65INJ5OD72TER2QMHS5X4W73TICIQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.906589640 (68h54m20.337669771s) [noise=true , address=104.36.82.36:4648 , piece_id=P5VRG5RQRIFD62ZKFQKEU5G26ZZTHNCSTZDKMWWZPZQLMHRJDLGA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.906928439 (68h54m20.33800857s) [noise=true , address=104.36.82.12:4179 , piece_id=AYVPU6T2SK4UVWU3UZRUSAOJ2ZMBJQETACGIU5OM22MNZ2OHPHHQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.907268709 (68h54m20.33834884s) [noise=true , address=104.36.82.23:4462 , piece_id=H4SGRBHOVYFT2JJR5CZRPA6XZNCHQI2JQRSEUPW3ELSSILPD4MPA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.907587840 (68h54m20.338667961s) [noise=true , address=104.36.82.44:4883 , piece_id=EKLHXVWSJ7MNI6I5H6USYF62S5LNHHYB6COZ7YUADJNXOTSAXI2Q, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.907867826 (68h54m20.338947957s) [noise=true , address=64.49.13.26:4946 , piece_id=JEXHP2LGBFSBKA7CVUQRITO6XUHT7P4DJ447PQIVDCMKUZL6PXTA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.908126762 (68h54m20.339206893s) [noise=true , address=77.90.31.16:4755 , piece_id=4PYHK2D2AIOQOFWEEV6BJHSPSIDELTMMNM5UMX3FMOLJHMIHX6TA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.908465049 (68h54m20.33954519s) [noise=true , address=104.36.82.60:4360 , piece_id=4OVLWYTXQ5363F3AJZM2BHAJLNTQW6N4T4YRPVBO6YN3PXCBXGXQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.908660383 (68h54m20.339740514s) [noise=true , address=45.91.116.7:4028 , piece_id=WG2EAQHMXG75F7LSKS4H37C3LNDXFR72GPOGKF2I7EOWE6WRZYFA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.908973892 (68h54m20.340054023s) [noise=true , address=104.36.82.8:4002 , piece_id=SZOGFOMS7GLJU5HZYIXQJ6VUNFZJFIBPVQPWDH7TGEHYBEB5356A, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.909241053 (68h54m20.340321174s) [noise=true , address=104.36.82.61:4382 , piece_id=NFOT3ACQLIE5KYNFEOKIFEK3WAG6SS4A37VL2ZMVJSO2M7RK2MXQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.909474511 (68h54m20.340554642s) [noise=true , address=104.36.82.47:4536 , piece_id=W76ZZAMYVLEIVZXDSUYX7N6RLH3GCFXOTGIZ6BB6WJ6DMYY7NTIQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.909737062 (68h54m20.340817193s) [noise=true , address=104.36.82.18:4378 , piece_id=SVV27TDMAWEUAUH6VNOSTFJR7TTKID6PMHLAQ6Z5AHYUIBZH2RMA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:27.467274710 (68h54m21.898354841s) [upload=71821]: Next returned nil segment Aug 11 14:20:27.467291802 (68h54m21.898371933s) [upload=71821]: Waiting for error group managing segments... Aug 11 14:20:27.874733898 (68h54m22.305814069s) [noise=true , address=77.90.31.16:4755 , piece_id=4PYHK2D2AIOQOFWEEV6BJHSPSIDELTMMNM5UMX3FMOLJHMIHX6TA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:27.928151777 (68h54m22.359231908s) [noise=true , address=104.36.82.59:4307 , piece_id=WMLIRN56JTUYCVIFLK77B6T7LMWKOWJZEFEX4VMCKLPHX4PERMGQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:27.958541664 (68h54m22.389621795s) [noise=true , address=104.36.82.18:4378 , piece_id=SVV27TDMAWEUAUH6VNOSTFJR7TTKID6PMHLAQ6Z5AHYUIBZH2RMA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:27.968795743 (68h54m22.399875874s) [noise=true , address=104.36.82.61:4382 , piece_id=NFOT3ACQLIE5KYNFEOKIFEK3WAG6SS4A37VL2ZMVJSO2M7RK2MXQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:27.971377493 (68h54m22.402457624s) [noise=true , address=104.36.82.44:4883 , piece_id=EKLHXVWSJ7MNI6I5H6USYF62S5LNHHYB6COZ7YUADJNXOTSAXI2Q, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.010470913 (68h54m22.441551044s) [noise=true , address=104.36.82.23:4462 , piece_id=H4SGRBHOVYFT2JJR5CZRPA6XZNCHQI2JQRSEUPW3ELSSILPD4MPA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.115715955 (68h54m22.546796086s) [noise=true , address=64.49.13.26:4946 , piece_id=JEXHP2LGBFSBKA7CVUQRITO6XUHT7P4DJ447PQIVDCMKUZL6PXTA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.466365867 (68h54m22.897445998s) [noise=true , address=104.36.82.8:4002 , piece_id=SZOGFOMS7GLJU5HZYIXQJ6VUNFZJFIBPVQPWDH7TGEHYBEB5356A, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.590496044 (68h54m23.021576175s) [noise=true , address=104.36.82.36:4648 , piece_id=P5VRG5RQRIFD62ZKFQKEU5G26ZZTHNCSTZDKMWWZPZQLMHRJDLGA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.601156390 (68h54m23.032236521s) [noise=true , address=104.36.82.47:4536 , piece_id=W76ZZAMYVLEIVZXDSUYX7N6RLH3GCFXOTGIZ6BB6WJ6DMYY7NTIQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.643327799 (68h54m23.07440793s) [noise=true , address=77.90.31.13:4639 , piece_id=2TR2AB4NLZTCIUDAFIZEYR5W3BUBSVPAMSOKNX32HQ6527KDXG7Q, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.698485266 (68h54m23.129565397s) [noise=true , address=104.36.82.12:4179 , piece_id=AYVPU6T2SK4UVWU3UZRUSAOJ2ZMBJQETACGIU5OM22MNZ2OHPHHQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.746588125 (68h54m23.177668256s) [noise=true , address=104.36.82.23:4451 , piece_id=3RWYSPA5GZY5BXSAVP45WMHNTWL6KLL5MPK2QFCZX6MMYUDOX3SQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:29.099681692 (68h54m23.530761813s) [noise=true , address=45.91.116.7:4028 , piece_id=WG2EAQHMXG75F7LSKS4H37C3LNDXFR72GPOGKF2I7EOWE6WRZYFA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:29.099763879 (68h54m23.53084401s) [seg_pos={0 0}, upload=71821]: Segment reached optimal threshold of 14 pieces. Aug 11 14:20:29.100020019 (68h54m23.53110015s) [noise=true , address=104.36.82.60:4360 , piece_id=4OVLWYTXQ5363F3AJZM2BHAJLNTQW6N4T4YRPVBO6YN3PXCBXGXQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T24rtYBJ47E6Ef1ZZKMWcpBrWBpCg5QV2ZHniCZZkzJF2yUNk): protocol: expected piece hash; context canceled; EOF Aug 11 14:20:29.100127134 (68h54m23.531207265s) [noise=true , address=104.36.82.45:4920 , piece_id=BKPN6F3FSG7Y2WZS5EUYUTS65INJ5OD72TER2QMHS5X4W73TICIQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ryY6XP6e8yUpMgwUmwf5z5syrjj6jBqzMpt2AUL9yV8tbd18S): protocol: expected piece hash; context canceled; EOF Aug 11 14:20:29.100174885 (68h54m23.531255006s) [noise=true , address=104.36.82.5:4851 , piece_id=CNDTEW4A5HF3ZV44ZSS3GIXDOB6SEI7HFCB3OVZJLSBILCKEFSMA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fTtTieEUquaKNUNzqK98PkX7tVxjsiNeMLjz8kQJwdLne7d9R): protocol: write tcp 188.166.225.170:54712->104.36.82.5:4851: use of closed network connection; write tcp 188.166.225.170:54712->104.36.82.5:4851: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54712->104.36.82.5:4851: use of closed network connection Aug 11 14:20:29.100247985 (68h54m23.531328126s) [noise=true , address=104.36.82.35:4605 , piece_id=77ER5KLFOPIVKQWZMTYQV3GAA7ASMKQDD5AXNCZ3HTZOR36O2KXQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12s1YRbSeZsYjBeTEBGb47tpitCD1hHdnC1Ysr2iUDvm1Wh6aDm): protocol: expected piece hash; context canceled; EOF Aug 11 14:20:29.100283202 (68h54m23.531363333s) [seg_pos={0 0}, upload=71821]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:20:29.100409504 (68h54m23.531489635s) [upload=71821]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:20:29.100418110 (68h54m23.531498241s) [upload=71821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:20:29.585811325 (68h54m24.016891456s) [upload=71821]: Done uploading segments... Aug 11 14:20:29.585827677 (68h54m24.016907808s) [upload=71821]: Upload finished. err: <nil> Duration: 7.083747856s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......X................X......X................X............] duration: 1.830841839s FAIL (2025-08-11 14:21:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c6b9157345243272a7fd8b4dc098f37cceeab798e6cdf11d954d6ff9ef48c84e != before 46f4349504aecf5ff98fe6ccb68a700449d3f8f9e38419adb777af313d841b65 ======LOGS====== Aug 11 14:10:49.640044296 (68h44m44.071124427s) [upload=71659]: Starting upload Aug 11 14:10:49.640093711 (68h44m44.071173842s) [upload=71659]: Done starting upload Aug 11 14:10:49.640145270 (68h44m44.071225401s) [upload=71659]: Uploading segments... Aug 11 14:10:49.640188944 (68h44m44.071269075s) [upload=71659]: Got next segment. Inline: true Aug 11 14:10:49.640192851 (68h44m44.071272982s) [upload=71659]: Waiting for error group managing segments... Aug 11 14:10:49.640224531 (68h44m44.071304662s) [upload=71659]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:10:49.640228519 (68h44m44.07130865s) [upload=71659]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:10:49.640230823 (68h44m44.071310954s) [upload=71659]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:10:50.122416853 (68h44m44.553496984s) [upload=71659]: Done uploading segments... Aug 11 14:10:50.122432393 (68h44m44.553512524s) [upload=71659]: Upload finished. err: <nil> Duration:* 1.830867317s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X......X..................X..............................X] duration: 5.233789672s FAIL (2025-08-11 14:20:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 14:20:25.453217472 (68h54m19.884297604s) [upload=71821]: Starting upload Aug 11 14:20:25.453256097 (68h54m19.884336228s) [upload=71821]: Done starting upload Aug 11 14:20:25.453429559 (68h54m19.88450969s) [upload=71821]: Uploading segments... Aug 11 14:20:25.453462020 (68h54m19.884542152s) [upload=71821]: Got next segment. Inline: false Aug 11 14:20:25.453484393 (68h54m19.884564525s) [upload=71821]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:20:25.453488852 (68h54m19.884568973s) [upload=71821]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 14:20:25.904819974 (68h54m20.335900105s) [seg_pos={0 0}, upload=71821]: Begin upload segment... Aug 11 14:20:25.904919054 (68h54m20.335999185s) [seg_pos={0 0}, upload=71821]: Done begin upload segment. Aug 11 14:20:25.904942519 (68h54m20.33602265s) [noise=true , address=104.36.82.59:4307 , piece_id=WMLIRN56JTUYCVIFLK77B6T7LMWKOWJZEFEX4VMCKLPHX4PERMGQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.904966685 (68h54m20.336046806s) [noise=true , address=77.90.31.13:4639 , piece_id=2TR2AB4NLZTCIUDAFIZEYR5W3BUBSVPAMSOKNX32HQ6527KDXG7Q, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.905322626 (68h54m20.336402747s) [noise=true , address=104.36.82.35:4605 , piece_id=77ER5KLFOPIVKQWZMTYQV3GAA7ASMKQDD5AXNCZ3HTZOR36O2KXQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.905450511 (68h54m20.336530632s) [noise=true , address=104.36.82.5:4851 , piece_id=CNDTEW4A5HF3ZV44ZSS3GIXDOB6SEI7HFCB3OVZJLSBILCKEFSMA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.905619344 (68h54m20.336699475s) [noise=true , address=104.36.82.23:4451 , piece_id=3RWYSPA5GZY5BXSAVP45WMHNTWL6KLL5MPK2QFCZX6MMYUDOX3SQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.905963603 (68h54m20.337043734s) [noise=true , address=104.36.82.45:4920 , piece_id=BKPN6F3FSG7Y2WZS5EUYUTS65INJ5OD72TER2QMHS5X4W73TICIQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.906589640 (68h54m20.337669771s) [noise=true , address=104.36.82.36:4648 , piece_id=P5VRG5RQRIFD62ZKFQKEU5G26ZZTHNCSTZDKMWWZPZQLMHRJDLGA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.906928439 (68h54m20.33800857s) [noise=true , address=104.36.82.12:4179 , piece_id=AYVPU6T2SK4UVWU3UZRUSAOJ2ZMBJQETACGIU5OM22MNZ2OHPHHQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.907268709 (68h54m20.33834884s) [noise=true , address=104.36.82.23:4462 , piece_id=H4SGRBHOVYFT2JJR5CZRPA6XZNCHQI2JQRSEUPW3ELSSILPD4MPA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.907587840 (68h54m20.338667961s) [noise=true , address=104.36.82.44:4883 , piece_id=EKLHXVWSJ7MNI6I5H6USYF62S5LNHHYB6COZ7YUADJNXOTSAXI2Q, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.907867826 (68h54m20.338947957s) [noise=true , address=64.49.13.26:4946 , piece_id=JEXHP2LGBFSBKA7CVUQRITO6XUHT7P4DJ447PQIVDCMKUZL6PXTA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.908126762 (68h54m20.339206893s) [noise=true , address=77.90.31.16:4755 , piece_id=4PYHK2D2AIOQOFWEEV6BJHSPSIDELTMMNM5UMX3FMOLJHMIHX6TA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.908465049 (68h54m20.33954519s) [noise=true , address=104.36.82.60:4360 , piece_id=4OVLWYTXQ5363F3AJZM2BHAJLNTQW6N4T4YRPVBO6YN3PXCBXGXQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.908660383 (68h54m20.339740514s) [noise=true , address=45.91.116.7:4028 , piece_id=WG2EAQHMXG75F7LSKS4H37C3LNDXFR72GPOGKF2I7EOWE6WRZYFA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.908973892 (68h54m20.340054023s) [noise=true , address=104.36.82.8:4002 , piece_id=SZOGFOMS7GLJU5HZYIXQJ6VUNFZJFIBPVQPWDH7TGEHYBEB5356A, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.909241053 (68h54m20.340321174s) [noise=true , address=104.36.82.61:4382 , piece_id=NFOT3ACQLIE5KYNFEOKIFEK3WAG6SS4A37VL2ZMVJSO2M7RK2MXQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.909474511 (68h54m20.340554642s) [noise=true , address=104.36.82.47:4536 , piece_id=W76ZZAMYVLEIVZXDSUYX7N6RLH3GCFXOTGIZ6BB6WJ6DMYY7NTIQ, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:25.909737062 (68h54m20.340817193s) [noise=true , address=104.36.82.18:4378 , piece_id=SVV27TDMAWEUAUH6VNOSTFJR7TTKID6PMHLAQ6Z5AHYUIBZH2RMA, seg_pos={0 0}, upload=71821]: Uploading piece... Aug 11 14:20:27.467274710 (68h54m21.898354841s) [upload=71821]: Next returned nil segment Aug 11 14:20:27.467291802 (68h54m21.898371933s) [upload=71821]: Waiting for error group managing segments... Aug 11 14:20:27.874733898 (68h54m22.305814069s) [noise=true , address=77.90.31.16:4755 , piece_id=4PYHK2D2AIOQOFWEEV6BJHSPSIDELTMMNM5UMX3FMOLJHMIHX6TA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:27.928151777 (68h54m22.359231908s) [noise=true , address=104.36.82.59:4307 , piece_id=WMLIRN56JTUYCVIFLK77B6T7LMWKOWJZEFEX4VMCKLPHX4PERMGQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:27.958541664 (68h54m22.389621795s) [noise=true , address=104.36.82.18:4378 , piece_id=SVV27TDMAWEUAUH6VNOSTFJR7TTKID6PMHLAQ6Z5AHYUIBZH2RMA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:27.968795743 (68h54m22.399875874s) [noise=true , address=104.36.82.61:4382 , piece_id=NFOT3ACQLIE5KYNFEOKIFEK3WAG6SS4A37VL2ZMVJSO2M7RK2MXQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:27.971377493 (68h54m22.402457624s) [noise=true , address=104.36.82.44:4883 , piece_id=EKLHXVWSJ7MNI6I5H6USYF62S5LNHHYB6COZ7YUADJNXOTSAXI2Q, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.010470913 (68h54m22.441551044s) [noise=true , address=104.36.82.23:4462 , piece_id=H4SGRBHOVYFT2JJR5CZRPA6XZNCHQI2JQRSEUPW3ELSSILPD4MPA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.115715955 (68h54m22.546796086s) [noise=true , address=64.49.13.26:4946 , piece_id=JEXHP2LGBFSBKA7CVUQRITO6XUHT7P4DJ447PQIVDCMKUZL6PXTA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.466365867 (68h54m22.897445998s) [noise=true , address=104.36.82.8:4002 , piece_id=SZOGFOMS7GLJU5HZYIXQJ6VUNFZJFIBPVQPWDH7TGEHYBEB5356A, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.590496044 (68h54m23.021576175s) [noise=true , address=104.36.82.36:4648 , piece_id=P5VRG5RQRIFD62ZKFQKEU5G26ZZTHNCSTZDKMWWZPZQLMHRJDLGA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.601156390 (68h54m23.032236521s) [noise=true , address=104.36.82.47:4536 , piece_id=W76ZZAMYVLEIVZXDSUYX7N6RLH3GCFXOTGIZ6BB6WJ6DMYY7NTIQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.643327799 (68h54m23.07440793s) [noise=true , address=77.90.31.13:4639 , piece_id=2TR2AB4NLZTCIUDAFIZEYR5W3BUBSVPAMSOKNX32HQ6527KDXG7Q, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.698485266 (68h54m23.129565397s) [noise=true , address=104.36.82.12:4179 , piece_id=AYVPU6T2SK4UVWU3UZRUSAOJ2ZMBJQETACGIU5OM22MNZ2OHPHHQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:28.746588125 (68h54m23.177668256s) [noise=true , address=104.36.82.23:4451 , piece_id=3RWYSPA5GZY5BXSAVP45WMHNTWL6KLL5MPK2QFCZX6MMYUDOX3SQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:29.099681692 (68h54m23.530761813s) [noise=true , address=45.91.116.7:4028 , piece_id=WG2EAQHMXG75F7LSKS4H37C3LNDXFR72GPOGKF2I7EOWE6WRZYFA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: <nil> Aug 11 14:20:29.099763879 (68h54m23.53084401s) [seg_pos={0 0}, upload=71821]: Segment reached optimal threshold of 14 pieces. Aug 11 14:20:29.100020019 (68h54m23.53110015s) [noise=true , address=104.36.82.60:4360 , piece_id=4OVLWYTXQ5363F3AJZM2BHAJLNTQW6N4T4YRPVBO6YN3PXCBXGXQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T24rtYBJ47E6Ef1ZZKMWcpBrWBpCg5QV2ZHniCZZkzJF2yUNk): protocol: expected piece hash; context canceled; EOF Aug 11 14:20:29.100127134 (68h54m23.531207265s) [noise=true , address=104.36.82.45:4920 , piece_id=BKPN6F3FSG7Y2WZS5EUYUTS65INJ5OD72TER2QMHS5X4W73TICIQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ryY6XP6e8yUpMgwUmwf5z5syrjj6jBqzMpt2AUL9yV8tbd18S): protocol: expected piece hash; context canceled; EOF Aug 11 14:20:29.100174885 (68h54m23.531255006s) [noise=true , address=104.36.82.5:4851 , piece_id=CNDTEW4A5HF3ZV44ZSS3GIXDOB6SEI7HFCB3OVZJLSBILCKEFSMA, seg_pos={0 0}, upload=71821]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fTtTieEUquaKNUNzqK98PkX7tVxjsiNeMLjz8kQJwdLne7d9R): protocol: write tcp 188.166.225.170:54712->104.36.82.5:4851: use of closed network connection; write tcp 188.166.225.170:54712->104.36.82.5:4851: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54712->104.36.82.5:4851: use of closed network connection Aug 11 14:20:29.100247985 (68h54m23.531328126s) [noise=true , address=104.36.82.35:4605 , piece_id=77ER5KLFOPIVKQWZMTYQV3GAA7ASMKQDD5AXNCZ3HTZOR36O2KXQ, seg_pos={0 0}, upload=71821]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12s1YRbSeZsYjBeTEBGb47tpitCD1hHdnC1Ysr2iUDvm1Wh6aDm): protocol: expected piece hash; context canceled; EOF Aug 11 14:20:29.100283202 (68h54m23.531363333s) [seg_pos={0 0}, upload=71821]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 14:20:29.100409504 (68h54m23.531489635s) [upload=71821]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 14:20:29.100418110 (68h54m23.531498241s) [upload=71821]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:20:29.585811325 (68h54m24.016891456s) [upload=71821]: Done uploading segments... Aug 11 14:20:29.585827677 (68h54m24.016907808s) [upload=71821]: Upload finished. err: <nil> Duration: 5.233854627s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.234914961s FAIL (2025-08-11 14:20:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.234940902s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......X................X......X................X...........] duration: 1.896438529s FAIL (2025-08-11 14:20:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c6b9157345243272a7fd8b4dc098f37cceeab798e6cdf11d954d6ff9ef48c84e != before 46f4349504aecf5ff98fe6ccb68a700449d3f8f9e38419adb777af313d841b65 ======LOGS====== Aug 11 14:10:49.640044296 (68h44m44.071124427s) [upload=71659]: Starting upload Aug 11 14:10:49.640093711 (68h44m44.071173842s) [upload=71659]: Done starting upload Aug 11 14:10:49.640145270 (68h44m44.071225401s) [upload=71659]: Uploading segments... Aug 11 14:10:49.640188944 (68h44m44.071269075s) [upload=71659]: Got next segment. Inline: true Aug 11 14:10:49.640192851 (68h44m44.071272982s) [upload=71659]: Waiting for error group managing segments... Aug 11 14:10:49.640224531 (68h44m44.071304662s) [upload=71659]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:10:49.640228519 (68h44m44.07130865s) [upload=71659]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:10:49.640230823 (68h44m44.071310954s) [upload=71659]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:10:50.122416853 (68h44m44.553496984s) [upload=71659]: Done uploading segments... Aug 11 14:10:50.122432393 (68h44m44.553512524s) [upload=71659]: Upload finished. err: <nil> Duration:* 1.896474889s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X......X..................X..............................] duration: 6.618098518s FAIL (2025-08-11 14:19:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8b55aea5df265895cd87ccde5adf694bcfbd212f6792ccd2fb147e564b767c8f != before b549c75655828d2f55f4613a05deeffb0861c1c57fbf835829990cf0277b3fa1 ======LOGS====== Aug 11 13:56:58.224881633 (68h30m52.655961754s) [upload=71416]: Starting upload Aug 11 13:56:58.224939675 (68h30m52.656019796s) [upload=71416]: Done starting upload Aug 11 13:56:58.225048664 (68h30m52.656128795s) [upload=71416]: Uploading segments... Aug 11 13:56:58.225132684 (68h30m52.656212805s) [upload=71416]: Got next segment. Inline: false Aug 11 13:56:58.225160648 (68h30m52.656240779s) [upload=71416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 13:56:58.225164916 (68h30m52.656245047s) [upload=71416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 13:56:58.696709334 (68h30m53.127789465s) [seg_pos={0 0}, upload=71416]: Begin upload segment... Aug 11 13:56:58.696752567 (68h30m53.127832699s) [seg_pos={0 0}, upload=71416]: Done begin upload segment. Aug 11 13:56:58.696774159 (68h30m53.12785429s) [noise=true , address=104.36.82.37:4662 , piece_id=D7TMYWLEB7LDPYUALRPSCDDI4D56PMEKMTHPH36MYZSE3SY7R3NA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.696838402 (68h30m53.127918523s) [noise=true , address=104.36.82.6:4898 , piece_id=Z7SRLD63PZWXOCC6TVWENFWEL4OFG2TUZFCWT3YVGNTRJX2WMHNA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697139798 (68h30m53.128219929s) [noise=true , address=104.36.82.18:4365 , piece_id=23J7XG3U67LW2YTBUVHWFQ4C3WCBGWHDIGDU4FJTOHMQAKLAUXTA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697343479 (68h30m53.1284236s) [noise=true , address=104.36.82.36:4622 , piece_id=TNV7G34ONP3KIULW2DFEQNJQLFAD7ELRXYU5DZQWNZ7R75N26WDQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697443419 (68h30m53.12852355s) [noise=true , address=64.49.13.155:4923 , piece_id=VQKAJYAPJBNTCEG6CBO6R4IGAPAX2UMXTRPXBD54IYGQFKK6U6ZA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697678049 (68h30m53.12875817s) [noise=true , address=104.36.82.50:4024 , piece_id=ZGUCYKRBVGZZ5OZ4ER2U6TBRKWGZ52ALEGCZIZ6YLPZKJXQEUYQA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697722623 (68h30m53.128802754s) [noise=true , address=104.36.82.23:4445 , piece_id=FJZMBDLBEXLNCRWJBZ254WFY2I3HSPFBFIQHJTLG7WGSOWOEPCJQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698059739 (68h30m53.12913987s) [noise=true , address=45.91.116.11:4135 , piece_id=T3DUZFDS25Z7R2NIEEYE4JNVXROVTWF3BKAQ2ZHKWT2N6XUNK6VQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698094145 (68h30m53.129174276s) [noise=true , address=104.36.82.47:4538 , piece_id=7XY63ZHOMV6L3BDN5R6DPERVYKQDCN5GWQSJLR6W2ORMD7SBMPJQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698293766 (68h30m53.129373897s) [noise=true , address=104.36.82.7:4921 , piece_id=ZNZBLXEFJNXQKQFEPJA27XC5JF2KUJM67XQAPUGWPPHW722Z5GGQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698384790 (68h30m53.129464911s) [noise=true , address=104.36.82.59:4330 , piece_id=2R5MOC7YVFJ7WZZAH4JCS3ABSP23HXH7UL2FQ26ZNTWVL6OYHUKA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698530149 (68h30m53.12961028s) [noise=true , address=104.36.82.61:4392 , piece_id=Y727CNHOLD4AI3IPOUTIBBSAXRTSE2Z6HVMWBO3JODNLL6QAJARQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698672501 (68h30m53.129752622s) [noise=true , address=104.36.82.38:4693 , piece_id=3R3RPFRP2ICJNGTTSPGBM22GCZIL3VVFUXCEDCJKA52XX2BPBJGA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698750641 (68h30m53.129830762s) [noise=true , address=104.36.82.62:4420 , piece_id=TFQ6H5F2BHAQ75HX6ZM2UTXPROFKID5S4RLDWDBAPCRFC2U5J5XA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698994197 (68h30m53.130074318s) [noise=true , address=193.149.164.143:4693 , piece_id=ICOKEV23JWZT6LLRHFFP7DHOKO7BEFNHFQLJ44HIZP53RQQIMD3Q, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.699025887 (68h30m53.130106018s) [noise=true , address=104.36.82.34:4566 , piece_id=4VZ36VJUMUGEBLT7IIMI5C7P7GP53GG5EC5FULKUM7JKX5C46K5Q, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.699242913 (68h30m53.130323034s) [noise=true , address=64.49.13.139:4477 , piece_id=DAA6LS6FX6DEBW5AL22SCUHDHD5MQUSSRWTALMS2CUBQWHM6BLXQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.699249946 (68h30m53.130330077s) [noise=true , address=104.36.82.60:4351 , piece_id=7GO7KIDCFWM6H5AVF4SKOJY6NU5UQRELUNA3G3NRV6WOJQFFIT5Q, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:59.855404318 (68h30m54.286484449s) [upload=71416]: Next returned nil segment Aug 11 13:56:59.855424016 (68h30m54.286504147s) [upload=71416]: Waiting for error group managing segments... Aug 11 13:57:00.152023922 (68h30m54.583104053s) [noise=true , address=64.49.13.139:4477 , piece_id=DAA6LS6FX6DEBW5AL22SCUHDHD5MQUSSRWTALMS2CUBQWHM6BLXQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.162733814 (68h30m54.593813955s) [noise=true , address=64.49.13.155:4923 , piece_id=VQKAJYAPJBNTCEG6CBO6R4IGAPAX2UMXTRPXBD54IYGQFKK6U6ZA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.356081019 (68h30m54.78716115s) [noise=true , address=193.149.164.143:4693 , piece_id=ICOKEV23JWZT6LLRHFFP7DHOKO7BEFNHFQLJ44HIZP53RQQIMD3Q, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.660126702 (68h30m55.091206833s) [noise=true , address=104.36.82.6:4898 , piece_id=Z7SRLD63PZWXOCC6TVWENFWEL4OFG2TUZFCWT3YVGNTRJX2WMHNA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.671620063 (68h30m55.102700194s) [noise=true , address=104.36.82.59:4330 , piece_id=2R5MOC7YVFJ7WZZAH4JCS3ABSP23HXH7UL2FQ26ZNTWVL6OYHUKA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.699652759 (68h30m55.13073289s) [noise=true , address=45.91.116.11:4135 , piece_id=T3DUZFDS25Z7R2NIEEYE4JNVXROVTWF3BKAQ2ZHKWT2N6XUNK6VQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.766271316 (68h30m55.197351447s) [noise=true , address=104.36.82.34:4566 , piece_id=4VZ36VJUMUGEBLT7IIMI5C7P7GP53GG5EC5FULKUM7JKX5C46K5Q, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.771038638 (68h30m55.202118759s) [noise=true , address=104.36.82.60:4351 , piece_id=7GO7KIDCFWM6H5AVF4SKOJY6NU5UQRELUNA3G3NRV6WOJQFFIT5Q, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.781180953 (68h30m55.212261084s) [noise=true , address=104.36.82.23:4445 , piece_id=FJZMBDLBEXLNCRWJBZ254WFY2I3HSPFBFIQHJTLG7WGSOWOEPCJQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.799690539 (68h30m55.23077067s) [noise=true , address=104.36.82.36:4622 , piece_id=TNV7G34ONP3KIULW2DFEQNJQLFAD7ELRXYU5DZQWNZ7R75N26WDQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.859141701 (68h30m55.290221832s) [noise=true , address=104.36.82.50:4024 , piece_id=ZGUCYKRBVGZZ5OZ4ER2U6TBRKWGZ52ALEGCZIZ6YLPZKJXQEUYQA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.887226417 (68h30m55.318306538s) [noise=true , address=104.36.82.37:4662 , piece_id=D7TMYWLEB7LDPYUALRPSCDDI4D56PMEKMTHPH36MYZSE3SY7R3NA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:01.574657599 (68h30m56.00573773s) [noise=true , address=104.36.82.62:4420 , piece_id=TFQ6H5F2BHAQ75HX6ZM2UTXPROFKID5S4RLDWDBAPCRFC2U5J5XA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:01.634497455 (68h30m56.065577587s) [noise=true , address=104.36.82.61:4392 , piece_id=Y727CNHOLD4AI3IPOUTIBBSAXRTSE2Z6HVMWBO3JODNLL6QAJARQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:01.634534466 (68h30m56.065614637s) [seg_pos={0 0}, upload=71416]: Segment reached optimal threshold of 14 pieces. Aug 11 13:57:01.634677259 (68h30m56.06575741s) [noise=true , address=104.36.82.7:4921 , piece_id=ZNZBLXEFJNXQKQFEPJA27XC5JF2KUJM67XQAPUGWPPHW722Z5GGQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QKJQDHC2wE9GjN8T92Ah33d6dyA2z3tqM1jg8tUp3J7B8cHww): protocol: expected piece hash; context canceled; EOF Aug 11 13:57:01.634731643 (68h30m56.065811774s) [noise=true , address=104.36.82.18:4365 , piece_id=23J7XG3U67LW2YTBUVHWFQ4C3WCBGWHDIGDU4FJTOHMQAKLAUXTA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq): protocol: expected piece hash; context canceled; EOF Aug 11 13:57:01.634771380 (68h30m56.065851511s) [noise=true , address=104.36.82.38:4693 , piece_id=3R3RPFRP2ICJNGTTSPGBM22GCZIL3VVFUXCEDCJKA52XX2BPBJGA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11E39ZZcpnJR43M2RU3Ggb8neyXj6bbSJkANxdJNGuZJiH8c9d): protocol: expected piece hash; context canceled; EOF Aug 11 13:57:01.634840622 (68h30m56.065920763s) [noise=true , address=104.36.82.47:4538 , piece_id=7XY63ZHOMV6L3BDN5R6DPERVYKQDCN5GWQSJLR6W2ORMD7SBMPJQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P3N79R3vpvkpsJ4fEd6NcJtmMpszRW5KHgfYJvkHQzvSTf3i5): protocol: expected piece hash; context canceled; EOF Aug 11 13:57:01.634858867 (68h30m56.065939028s) [seg_pos={0 0}, upload=71416]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 13:57:01.635002171 (68h30m56.066082332s) [upload=71416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 13:57:01.635010356 (68h30m56.066090517s) [upload=71416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 13:57:01.940262006 (68h30m56.371342138s) [upload=71416]: Done uploading segments... Aug 11 13:57:01.940287425 (68h30m56.371367556s) [upload=71416]: Upload finished. err: <nil> Duration: 6.618126742s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........X................X......X................X..........] duration: 2.019263398s FAIL (2025-08-11 14:19:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c6b9157345243272a7fd8b4dc098f37cceeab798e6cdf11d954d6ff9ef48c84e != before 46f4349504aecf5ff98fe6ccb68a700449d3f8f9e38419adb777af313d841b65 ======LOGS====== Aug 11 14:10:49.640044296 (68h44m44.071124427s) [upload=71659]: Starting upload Aug 11 14:10:49.640093711 (68h44m44.071173842s) [upload=71659]: Done starting upload Aug 11 14:10:49.640145270 (68h44m44.071225401s) [upload=71659]: Uploading segments... Aug 11 14:10:49.640188944 (68h44m44.071269075s) [upload=71659]: Got next segment. Inline: true Aug 11 14:10:49.640192851 (68h44m44.071272982s) [upload=71659]: Waiting for error group managing segments... Aug 11 14:10:49.640224531 (68h44m44.071304662s) [upload=71659]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:10:49.640228519 (68h44m44.07130865s) [upload=71659]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:10:49.640230823 (68h44m44.071310954s) [upload=71659]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:10:50.122416853 (68h44m44.553496984s) [upload=71659]: Done uploading segments... Aug 11 14:10:50.122432393 (68h44m44.553512524s) [upload=71659]: Upload finished. err: <nil> Duration:* 2.019293966s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.523753202s FAIL (2025-08-11 14:19:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.523780674s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X......X..................X.............................] duration: 6.696626117s FAIL (2025-08-11 14:18:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 8b55aea5df265895cd87ccde5adf694bcfbd212f6792ccd2fb147e564b767c8f != before b549c75655828d2f55f4613a05deeffb0861c1c57fbf835829990cf0277b3fa1 ======LOGS====== Aug 11 13:56:58.224881633 (68h30m52.655961754s) [upload=71416]: Starting upload Aug 11 13:56:58.224939675 (68h30m52.656019796s) [upload=71416]: Done starting upload Aug 11 13:56:58.225048664 (68h30m52.656128795s) [upload=71416]: Uploading segments... Aug 11 13:56:58.225132684 (68h30m52.656212805s) [upload=71416]: Got next segment. Inline: false Aug 11 13:56:58.225160648 (68h30m52.656240779s) [upload=71416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 13:56:58.225164916 (68h30m52.656245047s) [upload=71416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 13:56:58.696709334 (68h30m53.127789465s) [seg_pos={0 0}, upload=71416]: Begin upload segment... Aug 11 13:56:58.696752567 (68h30m53.127832699s) [seg_pos={0 0}, upload=71416]: Done begin upload segment. Aug 11 13:56:58.696774159 (68h30m53.12785429s) [noise=true , address=104.36.82.37:4662 , piece_id=D7TMYWLEB7LDPYUALRPSCDDI4D56PMEKMTHPH36MYZSE3SY7R3NA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.696838402 (68h30m53.127918523s) [noise=true , address=104.36.82.6:4898 , piece_id=Z7SRLD63PZWXOCC6TVWENFWEL4OFG2TUZFCWT3YVGNTRJX2WMHNA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697139798 (68h30m53.128219929s) [noise=true , address=104.36.82.18:4365 , piece_id=23J7XG3U67LW2YTBUVHWFQ4C3WCBGWHDIGDU4FJTOHMQAKLAUXTA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697343479 (68h30m53.1284236s) [noise=true , address=104.36.82.36:4622 , piece_id=TNV7G34ONP3KIULW2DFEQNJQLFAD7ELRXYU5DZQWNZ7R75N26WDQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697443419 (68h30m53.12852355s) [noise=true , address=64.49.13.155:4923 , piece_id=VQKAJYAPJBNTCEG6CBO6R4IGAPAX2UMXTRPXBD54IYGQFKK6U6ZA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697678049 (68h30m53.12875817s) [noise=true , address=104.36.82.50:4024 , piece_id=ZGUCYKRBVGZZ5OZ4ER2U6TBRKWGZ52ALEGCZIZ6YLPZKJXQEUYQA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.697722623 (68h30m53.128802754s) [noise=true , address=104.36.82.23:4445 , piece_id=FJZMBDLBEXLNCRWJBZ254WFY2I3HSPFBFIQHJTLG7WGSOWOEPCJQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698059739 (68h30m53.12913987s) [noise=true , address=45.91.116.11:4135 , piece_id=T3DUZFDS25Z7R2NIEEYE4JNVXROVTWF3BKAQ2ZHKWT2N6XUNK6VQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698094145 (68h30m53.129174276s) [noise=true , address=104.36.82.47:4538 , piece_id=7XY63ZHOMV6L3BDN5R6DPERVYKQDCN5GWQSJLR6W2ORMD7SBMPJQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698293766 (68h30m53.129373897s) [noise=true , address=104.36.82.7:4921 , piece_id=ZNZBLXEFJNXQKQFEPJA27XC5JF2KUJM67XQAPUGWPPHW722Z5GGQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698384790 (68h30m53.129464911s) [noise=true , address=104.36.82.59:4330 , piece_id=2R5MOC7YVFJ7WZZAH4JCS3ABSP23HXH7UL2FQ26ZNTWVL6OYHUKA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698530149 (68h30m53.12961028s) [noise=true , address=104.36.82.61:4392 , piece_id=Y727CNHOLD4AI3IPOUTIBBSAXRTSE2Z6HVMWBO3JODNLL6QAJARQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698672501 (68h30m53.129752622s) [noise=true , address=104.36.82.38:4693 , piece_id=3R3RPFRP2ICJNGTTSPGBM22GCZIL3VVFUXCEDCJKA52XX2BPBJGA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698750641 (68h30m53.129830762s) [noise=true , address=104.36.82.62:4420 , piece_id=TFQ6H5F2BHAQ75HX6ZM2UTXPROFKID5S4RLDWDBAPCRFC2U5J5XA, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.698994197 (68h30m53.130074318s) [noise=true , address=193.149.164.143:4693 , piece_id=ICOKEV23JWZT6LLRHFFP7DHOKO7BEFNHFQLJ44HIZP53RQQIMD3Q, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.699025887 (68h30m53.130106018s) [noise=true , address=104.36.82.34:4566 , piece_id=4VZ36VJUMUGEBLT7IIMI5C7P7GP53GG5EC5FULKUM7JKX5C46K5Q, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.699242913 (68h30m53.130323034s) [noise=true , address=64.49.13.139:4477 , piece_id=DAA6LS6FX6DEBW5AL22SCUHDHD5MQUSSRWTALMS2CUBQWHM6BLXQ, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:58.699249946 (68h30m53.130330077s) [noise=true , address=104.36.82.60:4351 , piece_id=7GO7KIDCFWM6H5AVF4SKOJY6NU5UQRELUNA3G3NRV6WOJQFFIT5Q, seg_pos={0 0}, upload=71416]: Uploading piece... Aug 11 13:56:59.855404318 (68h30m54.286484449s) [upload=71416]: Next returned nil segment Aug 11 13:56:59.855424016 (68h30m54.286504147s) [upload=71416]: Waiting for error group managing segments... Aug 11 13:57:00.152023922 (68h30m54.583104053s) [noise=true , address=64.49.13.139:4477 , piece_id=DAA6LS6FX6DEBW5AL22SCUHDHD5MQUSSRWTALMS2CUBQWHM6BLXQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.162733814 (68h30m54.593813955s) [noise=true , address=64.49.13.155:4923 , piece_id=VQKAJYAPJBNTCEG6CBO6R4IGAPAX2UMXTRPXBD54IYGQFKK6U6ZA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.356081019 (68h30m54.78716115s) [noise=true , address=193.149.164.143:4693 , piece_id=ICOKEV23JWZT6LLRHFFP7DHOKO7BEFNHFQLJ44HIZP53RQQIMD3Q, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.660126702 (68h30m55.091206833s) [noise=true , address=104.36.82.6:4898 , piece_id=Z7SRLD63PZWXOCC6TVWENFWEL4OFG2TUZFCWT3YVGNTRJX2WMHNA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.671620063 (68h30m55.102700194s) [noise=true , address=104.36.82.59:4330 , piece_id=2R5MOC7YVFJ7WZZAH4JCS3ABSP23HXH7UL2FQ26ZNTWVL6OYHUKA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.699652759 (68h30m55.13073289s) [noise=true , address=45.91.116.11:4135 , piece_id=T3DUZFDS25Z7R2NIEEYE4JNVXROVTWF3BKAQ2ZHKWT2N6XUNK6VQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.766271316 (68h30m55.197351447s) [noise=true , address=104.36.82.34:4566 , piece_id=4VZ36VJUMUGEBLT7IIMI5C7P7GP53GG5EC5FULKUM7JKX5C46K5Q, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.771038638 (68h30m55.202118759s) [noise=true , address=104.36.82.60:4351 , piece_id=7GO7KIDCFWM6H5AVF4SKOJY6NU5UQRELUNA3G3NRV6WOJQFFIT5Q, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.781180953 (68h30m55.212261084s) [noise=true , address=104.36.82.23:4445 , piece_id=FJZMBDLBEXLNCRWJBZ254WFY2I3HSPFBFIQHJTLG7WGSOWOEPCJQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.799690539 (68h30m55.23077067s) [noise=true , address=104.36.82.36:4622 , piece_id=TNV7G34ONP3KIULW2DFEQNJQLFAD7ELRXYU5DZQWNZ7R75N26WDQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.859141701 (68h30m55.290221832s) [noise=true , address=104.36.82.50:4024 , piece_id=ZGUCYKRBVGZZ5OZ4ER2U6TBRKWGZ52ALEGCZIZ6YLPZKJXQEUYQA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:00.887226417 (68h30m55.318306538s) [noise=true , address=104.36.82.37:4662 , piece_id=D7TMYWLEB7LDPYUALRPSCDDI4D56PMEKMTHPH36MYZSE3SY7R3NA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:01.574657599 (68h30m56.00573773s) [noise=true , address=104.36.82.62:4420 , piece_id=TFQ6H5F2BHAQ75HX6ZM2UTXPROFKID5S4RLDWDBAPCRFC2U5J5XA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:01.634497455 (68h30m56.065577587s) [noise=true , address=104.36.82.61:4392 , piece_id=Y727CNHOLD4AI3IPOUTIBBSAXRTSE2Z6HVMWBO3JODNLL6QAJARQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: <nil> Aug 11 13:57:01.634534466 (68h30m56.065614637s) [seg_pos={0 0}, upload=71416]: Segment reached optimal threshold of 14 pieces. Aug 11 13:57:01.634677259 (68h30m56.06575741s) [noise=true , address=104.36.82.7:4921 , piece_id=ZNZBLXEFJNXQKQFEPJA27XC5JF2KUJM67XQAPUGWPPHW722Z5GGQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QKJQDHC2wE9GjN8T92Ah33d6dyA2z3tqM1jg8tUp3J7B8cHww): protocol: expected piece hash; context canceled; EOF Aug 11 13:57:01.634731643 (68h30m56.065811774s) [noise=true , address=104.36.82.18:4365 , piece_id=23J7XG3U67LW2YTBUVHWFQ4C3WCBGWHDIGDU4FJTOHMQAKLAUXTA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rvMymD9ejLTfLTfeF4VAycp2SdGUMBtR3o45x8JeMkCDWpCHq): protocol: expected piece hash; context canceled; EOF Aug 11 13:57:01.634771380 (68h30m56.065851511s) [noise=true , address=104.36.82.38:4693 , piece_id=3R3RPFRP2ICJNGTTSPGBM22GCZIL3VVFUXCEDCJKA52XX2BPBJGA, seg_pos={0 0}, upload=71416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11E39ZZcpnJR43M2RU3Ggb8neyXj6bbSJkANxdJNGuZJiH8c9d): protocol: expected piece hash; context canceled; EOF Aug 11 13:57:01.634840622 (68h30m56.065920763s) [noise=true , address=104.36.82.47:4538 , piece_id=7XY63ZHOMV6L3BDN5R6DPERVYKQDCN5GWQSJLR6W2ORMD7SBMPJQ, seg_pos={0 0}, upload=71416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P3N79R3vpvkpsJ4fEd6NcJtmMpszRW5KHgfYJvkHQzvSTf3i5): protocol: expected piece hash; context canceled; EOF Aug 11 13:57:01.634858867 (68h30m56.065939028s) [seg_pos={0 0}, upload=71416]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 13:57:01.635002171 (68h30m56.066082332s) [upload=71416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 13:57:01.635010356 (68h30m56.066090517s) [upload=71416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 13:57:01.940262006 (68h30m56.371342138s) [upload=71416]: Done uploading segments... Aug 11 13:57:01.940287425 (68h30m56.371367556s) [upload=71416]: Upload finished. err: <nil> Duration: 6.696650875s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........X................X......X................X.........] duration: 2.174925968s FAIL (2025-08-11 14:18:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c6b9157345243272a7fd8b4dc098f37cceeab798e6cdf11d954d6ff9ef48c84e != before 46f4349504aecf5ff98fe6ccb68a700449d3f8f9e38419adb777af313d841b65 ======LOGS====== Aug 11 14:10:49.640044296 (68h44m44.071124427s) [upload=71659]: Starting upload Aug 11 14:10:49.640093711 (68h44m44.071173842s) [upload=71659]: Done starting upload Aug 11 14:10:49.640145270 (68h44m44.071225401s) [upload=71659]: Uploading segments... Aug 11 14:10:49.640188944 (68h44m44.071269075s) [upload=71659]: Got next segment. Inline: true Aug 11 14:10:49.640192851 (68h44m44.071272982s) [upload=71659]: Waiting for error group managing segments... Aug 11 14:10:49.640224531 (68h44m44.071304662s) [upload=71659]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 14:10:49.640228519 (68h44m44.07130865s) [upload=71659]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 14:10:49.640230823 (68h44m44.071310954s) [upload=71659]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 14:10:50.122416853 (68h44m44.553496984s) [upload=71659]: Done uploading segments... Aug 11 14:10:50.122432393 (68h44m44.553512524s) [upload=71659]: Upload finished. err: <nil> Duration:* 2.174951606s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 7.527179447s FAIL (2025-08-11 14:18:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.527360955s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)