Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.712082274s FAIL (2025-08-07 13:52:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.712107363s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.828810355s FAIL (2025-08-07 13:51:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.828837797s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.744589879s FAIL (2025-08-07 13:51:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.744615748s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.................X..................X.....................] duration: 5.887624922s FAIL (2025-08-07 13:51:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.887651363s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.629600632s FAIL (2025-08-07 13:51:27 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.629664114s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.................X..................X....................] duration: 5.890678531s FAIL (2025-08-07 13:51:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.890706496s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.111775395s FAIL (2025-08-07 13:50:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 2.111817284s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.753048002s FAIL (2025-08-07 13:50:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.753076196s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.899524652s FAIL (2025-08-07 13:50:20 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.899552826s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.................X..................X...................] duration: 5.986047683s FAIL (2025-08-07 13:50:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.986073231s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.968643153s FAIL (2025-08-07 13:49:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.968668132s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.862183831s FAIL (2025-08-07 13:49:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.862209531s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.998744869s FAIL (2025-08-07 13:49:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.998776931s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.................X..................X..................] duration: 6.003615274s FAIL (2025-08-07 13:49:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 6.003639019s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.118261349s FAIL (2025-08-07 13:48:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.118287018s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.................X..................X.................] duration: 5.738029332s FAIL (2025-08-07 13:48:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.738064208s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.676705963s FAIL (2025-08-07 13:48:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.676731642s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.963001251s FAIL (2025-08-07 13:48:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 2.963019787s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X.................X..................X................] duration: 5.919986278s FAIL (2025-08-07 13:47:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.920004874s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.901420043s FAIL (2025-08-07 13:47:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.901443728s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.860312053s FAIL (2025-08-07 13:47:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.860332823s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X.................X..................X...............] duration: 5.977308347s FAIL (2025-08-07 13:46:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.977337874s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.948516077s FAIL (2025-08-07 13:46:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.948537709s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.953610601s FAIL (2025-08-07 13:45:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.953628325s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X.................X..................X..............] duration: 5.687685042s FAIL (2025-08-07 13:45:50 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.687744616s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.020118969s FAIL (2025-08-07 13:45:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 2.020183172s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.783310275s FAIL (2025-08-07 13:45:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.783333449s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.868696649s FAIL (2025-08-07 13:44:48 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.868726496s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X.................X..................X.............] duration: 5.756090858s FAIL (2025-08-07 13:44:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.756117479s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 1.887888892s FAIL (2025-08-07 13:44:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.887909942s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X.................X..................X............] duration: 5.621482258s FAIL (2025-08-07 13:44:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.621507154s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.949898347s FAIL (2025-08-07 13:44:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.949922462s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.873830107s FAIL (2025-08-07 13:43:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.873859273s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.153634779s FAIL (2025-08-07 13:43:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 2.153658376s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X.................X..................X...........] duration: 6.004925114s FAIL (2025-08-07 13:43:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 6.004948108s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.991286252s FAIL (2025-08-07 13:43:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.991310197s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.849041363s FAIL (2025-08-07 13:42:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.849066521s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X.................X..................X..........] duration: 5.656189654s FAIL (2025-08-07 13:42:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.656205634s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.560366911s FAIL (2025-08-07 13:42:48 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.560391928s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X.................X..................X.........] duration: 5.664811427s FAIL (2025-08-07 13:42:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.66483422s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.025931314s FAIL (2025-08-07 13:42:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 2.025957755s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.821194932s FAIL (2025-08-07 13:41:48 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.821224568s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.744477354s FAIL (2025-08-07 13:41:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.74450198s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X.................X..................X........] duration: 6.032511812s FAIL (2025-08-07 13:41:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 6.032534767s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...................X.......................................] duration: 1.863585164s FAIL (2025-08-07 13:40:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.863613487s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.846520284s FAIL (2025-08-07 13:40:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.846541134s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X.................X..................X.......] duration: 5.877445516s FAIL (2025-08-07 13:40:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 5.877465877s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X...................X......................................] duration: 1.870464749s FAIL (2025-08-07 13:40:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 1.870487834s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X...................X.....................................] duration: 3.047059246s FAIL (2025-08-07 13:40:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 094400677dc524101fc94835b45685c5d30221124fb043d74c2fb0c1fd016d88 != before 99ad6dce77b959f2d478a93cd75079b6d92e7ceb9f1f9b1a07b956fd2bc9f4e0 ======LOGS====== tMakeInline Aug 7 13:04:50.432504137 (3h24m50.021849124s) [upload=3575]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:04:50.724668608 (3h24m50.314013595s) [upload=3575]: Done uploading segments... Aug 7 13:04:50.724686571 (3h24m50.314031548s) [upload=3575]: Upload finished. err: <nil> Aug 7 13:05:50.916838481 (3h25m50.506183468s) [upload=3593]: Starting upload Aug 7 13:05:50.916892374 (3h25m50.506237351s) [upload=3593]: Done starting upload Aug 7 13:05:50.916940717 (3h25m50.506285704s) [upload=3593]: Uploading segments... Aug 7 13:05:50.916974341 (3h25m50.506319328s) [upload=3593]: Got next segment. Inline: true Aug 7 13:05:50.916978689 (3h25m50.506323676s) [upload=3593]: Waiting for error group managing segments... Aug 7 13:05:50.917014678 (3h25m50.506359665s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:05:50.917021241 (3h25m50.506366218s) [upload=3593]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:05:50.917023676 (3h25m50.506368663s) [upload=3593]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:05:51.201153501 (3h25m50.790498488s) [upload=3593]: Done uploading segments... Aug 7 13:05:51.201177106 (3h25m50.790522093s) [upload=3593]: Upload finished. err: <nil> Aug 7 13:06:51.522697566 (3h26m51.112042554s) [upload=3609]: Starting upload Aug 7 13:06:51.522738174 (3h26m51.112083161s) [upload=3609]: Done starting upload Aug 7 13:06:51.522793469 (3h26m51.112138457s) [upload=3609]: Uploading segments... Aug 7 13:06:51.522819099 (3h26m51.112164086s) [upload=3609]: Got next segment. Inline: true Aug 7 13:06:51.522822335 (3h26m51.112167322s) [upload=3609]: Waiting for error group managing segments... Aug 7 13:06:51.522855018 (3h26m51.112200005s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:06:51.522858905 (3h26m51.112203892s) [upload=3609]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:06:51.522861230 (3h26m51.112206207s) [upload=3609]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:06:51.957295630 (3h26m51.546640617s) [upload=3609]: Done uploading segments... Aug 7 13:06:51.957315788 (3h26m51.546660775s) [upload=3609]: Upload finished. err: <nil> Aug 7 13:07:22.158240220 (3h27m21.747585207s) [upload=3619]: Starting upload Aug 7 13:07:22.158283853 (3h27m21.74762883s) [upload=3619]: Done starting upload Aug 7 13:07:22.158335221 (3h27m21.747680208s) [upload=3619]: Uploading segments... Aug 7 13:07:22.158405927 (3h27m21.747750904s) [upload=3619]: Got next segment. Inline: true Aug 7 13:07:22.158410225 (3h27m21.747755202s) [upload=3619]: Waiting for error group managing segments... Aug 7 13:07:22.158445923 (3h27m21.74779091s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:07:22.158450281 (3h27m21.747795268s) [upload=3619]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:07:22.158452396 (3h27m21.747797383s) [upload=3619]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:07:22.444567911 (3h27m22.033912888s) [upload=3619]: Done uploading segments... Aug 7 13:07:22.444585756 (3h27m22.033930743s) [upload=3619]: Upload finished. err: <nil> Aug 7 13:08:19.020722251 (3h28m18.610067239s) [upload=3636]: Starting upload Aug 7 13:08:19.020801012 (3h28m18.610145999s) [upload=3636]: Done starting upload Aug 7 13:08:19.020877910 (3h28m18.610222927s) [upload=3636]: Uploading segments... Aug 7 13:08:19.020933837 (3h28m18.610278854s) [upload=3636]: Got next segment. Inline: true Aug 7 13:08:19.020942974 (3h28m18.610287991s) [upload=3636]: Waiting for error group managing segments... Aug 7 13:08:19.021001315 (3h28m18.610346352s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:08:19.021008479 (3h28m18.610353506s) [upload=3636]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:08:19.021014059 (3h28m18.610359096s) [upload=3636]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:08:19.296409246 (3h28m18.885754233s) [upload=3636]: Done uploading segments... Aug 7 13:08:19.296439754 (3h28m18.885784741s) [upload=3636]: Upload finished. err: <nil> Aug 7 13:09:12.885582274 (3h29m12.474927271s) [upload=3651]: Starting upload Aug 7 13:09:12.885640766 (3h29m12.474985753s) [upload=3651]: Done starting upload Aug 7 13:09:12.885703456 (3h29m12.475048443s) [upload=3651]: Uploading segments... Aug 7 13:09:12.885741830 (3h29m12.475086817s) [upload=3651]: Got next segment. Inline: true Aug 7 13:09:12.885747330 (3h29m12.475092317s) [upload=3651]: Waiting for error group managing segments... Aug 7 13:09:12.885790552 (3h29m12.475135539s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:12.885794952 (3h29m12.475139939s) [upload=3651]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:12.885837022 (3h29m12.475182009s) [upload=3651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:13.154247394 (3h29m12.743592382s) [upload=3651]: Done uploading segments... Aug 7 13:09:13.154266291 (3h29m12.743611278s) [upload=3651]: Upload finished. err: <nil> Aug 7 13:09:44.054043531 (3h29m43.643388518s) [upload=3660]: Starting upload Aug 7 13:09:44.054097845 (3h29m43.643442832s) [upload=3660]: Done starting upload Aug 7 13:09:44.054153492 (3h29m43.643498479s) [upload=3660]: Uploading segments... Aug 7 13:09:44.054186705 (3h29m43.643531692s) [upload=3660]: Got next segment. Inline: true Aug 7 13:09:44.054191163 (3h29m43.64353615s) [upload=3660]: Waiting for error group managing segments... Aug 7 13:09:44.054225710 (3h29m43.643570687s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:09:44.054230299 (3h29m43.643575286s) [upload=3660]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:09:44.054232763 (3h29m43.643577751s) [upload=3660]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:09:44.652964288 (3h29m44.242309275s) [upload=3660]: Done uploading segments... Aug 7 13:09:44.652982053 (3h29m44.24232704s) [upload=3660]: Upload finished. err: <nil> Aug 7 13:10:32.365380708 (3h30m31.954725695s) [upload=3675]: Starting upload Aug 7 13:10:32.365446655 (3h30m31.954791642s) [upload=3675]: Done starting upload Aug 7 13:10:32.365502772 (3h30m31.954847759s) [upload=3675]: Uploading segments... Aug 7 13:10:32.365542718 (3h30m31.954887705s) [upload=3675]: Got next segment. Inline: true Aug 7 13:10:32.365547287 (3h30m31.954892264s) [upload=3675]: Waiting for error group managing segments... Aug 7 13:10:32.365585039 (3h30m31.954930026s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:10:32.365591542 (3h30m31.954936529s) [upload=3675]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:10:32.365596992 (3h30m31.954941989s) [upload=3675]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:10:32.672192162 (3h30m32.261537149s) [upload=3675]: Done uploading segments... Aug 7 13:10:32.672207491 (3h30m32.261552478s) [upload=3675]: Upload finished. err: <nil> Aug 7 13:11:42.828820500 (3h31m42.418165487s) [upload=3695]: Starting upload Aug 7 13:11:42.828861057 (3h31m42.418206044s) [upload=3695]: Done starting upload Aug 7 13:11:42.828903509 (3h31m42.418248496s) [upload=3695]: Uploading segments... Aug 7 13:11:42.828932424 (3h31m42.418277411s) [upload=3695]: Got next segment. Inline: true Aug 7 13:11:42.828935911 (3h31m42.418280888s) [upload=3695]: Waiting for error group managing segments... Aug 7 13:11:42.828966870 (3h31m42.418311857s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:11:42.828970386 (3h31m42.418315363s) [upload=3695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:11:42.828972981 (3h31m42.418317958s) [upload=3695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:11:43.394882119 (3h31m42.984227106s) [upload=3695]: Done uploading segments... Aug 7 13:11:43.394898771 (3h31m42.984243748s) [upload=3695]: Upload finished. err: <nil> Aug 7 13:12:14.735032230 (3h32m14.324377207s) [upload=3702]: Starting upload Aug 7 13:12:14.735081084 (3h32m14.324426071s) [upload=3702]: Done starting upload Aug 7 13:12:14.735134616 (3h32m14.324479613s) [upload=3702]: Uploading segments... Aug 7 13:12:14.735188078 (3h32m14.324533085s) [upload=3702]: Got next segment. Inline: true Aug 7 13:12:14.735193529 (3h32m14.324538526s) [upload=3702]: Waiting for error group managing segments... Aug 7 13:12:14.735311995 (3h32m14.324656992s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:14.735319549 (3h32m14.324664546s) [upload=3702]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:14.735323037 (3h32m14.324668034s) [upload=3702]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:15.008835749 (3h32m14.598180736s) [upload=3702]: Done uploading segments... Aug 7 13:12:15.008854536 (3h32m14.598199523s) [upload=3702]: Upload finished. err: <nil> Aug 7 13:12:34.045746707 (3h32m33.635091684s) [upload=3712]: Starting upload Aug 7 13:12:34.045823694 (3h32m33.635168681s) [upload=3712]: Done starting upload Aug 7 13:12:34.045881355 (3h32m33.635226342s) [upload=3712]: Uploading segments... Aug 7 13:12:34.045912053 (3h32m33.63525704s) [upload=3712]: Got next segment. Inline: true Aug 7 13:12:34.045915860 (3h32m33.635260847s) [upload=3712]: Waiting for error group managing segments... Aug 7 13:12:34.045950688 (3h32m33.635295675s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:12:34.045954565 (3h32m33.635299552s) [upload=3712]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 13:12:34.045956959 (3h32m33.635301946s) [upload=3712]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:12:34.346427481 (3h32m33.935772468s) [upload=3712]: Done uploading segments... Aug 7 13:12:34.346441938 (3h32m33.935786925s) [upload=3712]: Upload finished. err: <nil> Duration:* 3.047085427s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X.................X..................X......] duration: 6.647874237s FAIL (2025-08-07 13:39:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== oad=4095]: Done uploading segments... Aug 7 13:34:45.258440424 (3h54m44.847785411s) [upload=4095]: Upload finished. err: <nil> Aug 7 13:35:39.838446267 (3h55m39.427791254s) [upload=4113]: Starting upload Aug 7 13:35:39.838516742 (3h55m39.427861729s) [upload=4113]: Done starting upload Aug 7 13:35:39.838592306 (3h55m39.427937293s) [upload=4113]: Uploading segments... Aug 7 13:35:39.838789463 (3h55m39.428134521s) [upload=4113]: Got next segment. Inline: false Aug 7 13:35:39.838820573 (3h55m39.42816556s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 13:35:39.838824842 (3h55m39.428169829s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 13:35:40.304012890 (3h55m39.893357867s) [seg_pos={0 0}, upload=4113]: Begin upload segment... Aug 7 13:35:40.304077564 (3h55m39.893422551s) [seg_pos={0 0}, upload=4113]: Done begin upload segment. Aug 7 13:35:40.304098894 (3h55m39.893443881s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304111780 (3h55m39.893456767s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304443845 (3h55m39.893788832s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304696458 (3h55m39.894041445s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.304924995 (3h55m39.894269982s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305174813 (3h55m39.8945198s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305437296 (3h55m39.894782273s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305716700 (3h55m39.895061687s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.305892015 (3h55m39.895237002s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306168594 (3h55m39.895513581s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306395790 (3h55m39.895740767s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306690723 (3h55m39.89603571s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.306957945 (3h55m39.896302922s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307248190 (3h55m39.896593177s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307573332 (3h55m39.896918319s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307689895 (3h55m39.897034883s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.307936016 (3h55m39.897281003s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:40.308226622 (3h55m39.897571609s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Uploading piece... Aug 7 13:35:41.784121393 (3h55m41.373466381s) [upload=4113]: Next returned nil segment Aug 7 13:35:41.784136422 (3h55m41.373481409s) [upload=4113]: Waiting for error group managing segments... Aug 7 13:35:42.227746489 (3h55m41.817091476s) [noise=true , address=104.36.82.44:4887 , piece_id=2O6JTWFDRAO2RPN4Q35XDNBXTLYYR5KHS5BBZ3I76OPJR7BPWAXA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.229012962 (3h55m41.818357949s) [noise=true , address=104.36.82.37:4660 , piece_id=ZOKYHNCOQGK35UZ3X6E7B7TIANPPGEEUP6XJ5H6J7FG6337P6JGA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.252652088 (3h55m41.841997075s) [noise=true , address=104.36.82.53:4128 , piece_id=KBNXRMABLQWIHEOGEZNTKJA7FRXV4XN5ZD6Z76RP34ZOVX36DVYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.259550454 (3h55m41.848895441s) [noise=true , address=104.36.82.15:4283 , piece_id=TXNLF73VBPZ7RRSDB4Q6OCS676XSGWGU5Y4RCUALVWKX5XQYTIIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.268637960 (3h55m41.857982937s) [noise=true , address=104.36.82.56:4234 , piece_id=LFSVBMTTPDIEFUGSYZLGUFLOXQHTMBMZA4DLCHHG5EHAFZXU5PPQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.305217222 (3h55m41.894562249s) [noise=true , address=104.36.82.61:4408 , piece_id=33DEID7HHJ6ZWZWREX2TGFLGIKRXNBIQDQ7UKJI3LZAHR7D2TQYQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.311206029 (3h55m41.900551016s) [noise=true , address=104.36.82.35:4596 , piece_id=HXRZKGLQAU4CJBUY5PDXBW7TRUXRNI4QFM7BINGPA7U7KXPWA2EA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.318586579 (3h55m41.907931596s) [noise=true , address=104.36.82.59:4306 , piece_id=OD266OVKKMYQ44AFOB72QEW2UFSCYSCSVXVI4CX4GKULIQ6QXRWA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.324444155 (3h55m41.913789142s) [noise=true , address=104.36.82.23:4453 , piece_id=KXGUFLUKGXK2NKM6DZUWPPNPEMYLFB3MNOIQPBMX7MFC3LTXCITA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.351878150 (3h55m41.941223137s) [noise=true , address=104.36.82.55:4211 , piece_id=AYTUMUWXW55M2NVJAP5Y6I2KKIV3TNBR7P5MUZ72PMEMRNJ7RWIA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.363119627 (3h55m41.952464614s) [noise=true , address=104.36.82.45:4927 , piece_id=YVIQ7SVGS7OAGVVX5A5F57K57DQDRLPX5CCJ44EY2OML6OCP6SCA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.370849234 (3h55m41.960194222s) [noise=true , address=104.36.82.9:4008 , piece_id=OSIXKDZTB33WJYQMNLE6OOGAJIBKZCATXCO7UD2TURN4FEDVC4UA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.373899731 (3h55m41.963244708s) [noise=true , address=104.36.82.54:4191 , piece_id=ZLEYZSXEQ7YIHANVR4YLLSL74OMYRMQOACDG4P76DA6GXYUEWUQA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380846100 (3h55m41.970191077s) [noise=true , address=104.36.82.60:4343 , piece_id=ZFSDQXPZGD3Z4TZG5XOIFKRVA3J7HACYR2L5RBZYZKAPBL6DR5VA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: <nil> Aug 7 13:35:42.380858805 (3h55m41.970203792s) [seg_pos={0 0}, upload=4113]: Segment reached optimal threshold of 14 pieces. Aug 7 13:35:42.380950269 (3h55m41.970295257s) [noise=true , address=104.36.82.47:4520 , piece_id=QU4EVOYBZJIYIH4HLO7KJZDQ5Q3VS2GLUQJPMIIIYRAOLWXGHMHQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eNRsboM5zFiyBVPLuQfgb5QSAc5bZFizK9KJHudNd9nstVGyy): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.380980197 (3h55m41.970325184s) [noise=true , address=45.91.116.6:4002 , piece_id=HM5JNQXY2YAM2HYCEAFGI3LIFIZ7ADZL537T2Q7BZXLWYMHEMA2A, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FYX3Kj1Te1CZCEq81YVnKczjy9cy8e7RWzTJft2jwae6ZvaeY): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381007449 (3h55m41.970352436s) [noise=true , address=104.36.82.59:4311 , piece_id=KNKPKHASYPY2EOKESJKHOB2JEUZSLUWYEGL5Q22ZMMUMKOG5K4EQ, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bowY9sNioETZodrJQZF4P9evfk5fv82gT8nTEq3kvjMYi4buX): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381052255 (3h55m41.970397242s) [noise=true , address=104.36.82.55:4221 , piece_id=NX635KK3JCQ4JIDOFZRE7PEXM2NTNVWREIWTTEWJQ3C3IJUMNVYA, seg_pos={0 0}, upload=4113]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ssFiXk5oKTgnVMJ5TBJuSXuD1D2C46gT2hcY6yaDxf8fL1cjm): protocol: expected piece hash; context canceled; EOF Aug 7 13:35:42.381075268 (3h55m41.970420255s) [seg_pos={0 0}, upload=4113]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 13:35:42.381157105 (3h55m41.970502092s) [upload=4113]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 13:35:42.381161334 (3h55m41.970506321s) [upload=4113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 13:35:42.692877550 (3h55m42.282222537s) [upload=4113]: Done uploading segments... Aug 7 13:35:42.692895734 (3h55m42.282240722s) [upload=4113]: Upload finished. err: <nil> Duration: 6.647896038s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)