Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X...X.....X.............................................] duration: 8.790404477s FAIL (2025-08-12 13:13:28 UTC) orbiter warning: context deadline exceeded Duration: 8.790419326s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X.X..] duration: 1.872204814s FAIL (2025-08-12 13:13:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after e0afea1286d500709c6eacbc885f14f7096b14f8840e69c9d8166cb8e2debf29 != before 71329f9a8409421df671902efc6f0a67ff5f48d20c74cf8dedf1607d6cf675b6 ======LOGS====== Aug 12 13:11:35.965632559 (91h45m30.39671269s) [upload=95618]: Starting upload Aug 12 13:11:35.965698475 (91h45m30.396778606s) [upload=95618]: Done starting upload Aug 12 13:11:35.965761024 (91h45m30.396841155s) [upload=95618]: Uploading segments... Aug 12 13:11:35.965844103 (91h45m30.396924234s) [upload=95618]: Got next segment. Inline: true Aug 12 13:11:35.965854404 (91h45m30.396934535s) [upload=95618]: Waiting for error group managing segments... Aug 12 13:11:35.965890853 (91h45m30.396970974s) [upload=95618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:11:35.965895442 (91h45m30.396975573s) [upload=95618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 13:11:35.965898257 (91h45m30.396978388s) [upload=95618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 13:11:36.243042387 (91h45m30.674122508s) [upload=95618]: Done uploading segments... Aug 12 13:11:36.243057095 (91h45m30.674137226s) [upload=95618]: Upload finished. err: <nil> Duration:* 1.872228791s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X.......X.........X] duration: 4.059344202s FAIL (2025-08-12 13:13:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 12 13:13:17.720585912 (91h47m12.151666043s) [upload=95648]: Starting upload Aug 12 13:13:17.720647500 (91h47m12.151727631s) [upload=95648]: Done starting upload Aug 12 13:13:17.720860737 (91h47m12.151940858s) [upload=95648]: Uploading segments... Aug 12 13:13:17.721505532 (91h47m12.152585663s) [upload=95648]: Got next segment. Inline: false Aug 12 13:13:17.721535569 (91h47m12.1526157s) [upload=95648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:13:17.721540318 (91h47m12.152620449s) [upload=95648]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:13:18.206426520 (91h47m12.637506671s) [seg_pos={0 0}, upload=95648]: Begin upload segment... Aug 12 13:13:18.206498427 (91h47m12.637578558s) [seg_pos={0 0}, upload=95648]: Done begin upload segment. Aug 12 13:13:18.206564614 (91h47m12.637644745s) [noise=true , address=104.36.82.46:4514 , piece_id=7XJNJTDXNNU6WZSXKRNNNVSJSZCGGX7SELNYAPAMYQTOBPA3K6MA, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.206572950 (91h47m12.637653081s) [noise=true , address=79.127.201.215:4355 , piece_id=WCGMSZJLQEROU2L4KMY36T47XVQ4GAURFEEVMKAABBBHW5A3LT4A, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.207077224 (91h47m12.638157355s) [noise=true , address=104.36.82.61:4383 , piece_id=T7LBPLCP5NNOIQ2PJQOAQ37OVL6HZSP6UPLFOCCB3ITFDPI2SN5A, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.207130106 (91h47m12.638210227s) [noise=true , address=66.23.193.175:4554 , piece_id=PLZMNRKECIZYURSGDBHNA3QNREEF6XVRHEZMD3JENBHRUWKCOW2A, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.207428186 (91h47m12.638508317s) [noise=true , address=104.36.82.18:4395 , piece_id=JXPUGWSK35FSUFZCMOO2HOBFFSJ3PZYCYQMI7RFM764DK7ZZYYJQ, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.207931829 (91h47m12.63901196s) [noise=true , address=104.36.82.36:4630 , piece_id=V3FZULDW3NWAZVPCO7XOOIZP2O6ZVGB3I3X2IYAJCJBZGTLYXQSA, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.208165517 (91h47m12.639245648s) [noise=true , address=64.49.13.140:4509 , piece_id=FMQ46KQDGN3MYI75QL52SJ3OZZZVEYKMDQ3BC5AXSH7DIMIEZC6A, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.208398272 (91h47m12.639478403s) [noise=true , address=104.36.82.5:4872 , piece_id=TQOMRWQOBESZ67DFENZ5REFSQJ3NL76YQ67SWWVLJT3LIV7L5K6Q, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.208482032 (91h47m12.639562163s) [noise=true , address=104.36.82.62:4414 , piece_id=SVNSZARBEUW5GWUMRZKWXOLZASV2T4FO24HTVULNM3CHMXFMCFWA, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.208729506 (91h47m12.639809637s) [noise=true , address=104.36.82.60:4340 , piece_id=GUJQGU7K3IP3HGQPU4HENOKBJGMN7XXUQ46X2FI5Z47UUBJODQXQ, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.211187810 (91h47m12.642267941s) [noise=true , address=104.36.82.23:4463 , piece_id=FQ2UL7RYK4R5IMJXW7E4UYOU3XVJZ7TIAL3L2LN4DNJ3NH6K4QCA, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.212063946 (91h47m12.643144077s) [noise=true , address=104.36.82.58:4267 , piece_id=GFDZW5JPUK7JW2PZVROBX3NZTDW7PCOLCVS2BTSDMNQBAMZOD2GA, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.213286104 (91h47m12.644366235s) [noise=true , address=104.36.82.4:4829 , piece_id=M7AERIUO746K46W6RVIG6D3SYO42KNNO6W7AUVLKPYPGROTFJDKQ, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.214730368 (91h47m12.645810499s) [noise=true , address=104.36.82.17:4357 , piece_id=VYTJGJ4EAN6SA55WYMHPXA5FNXYDAWHIQTFBSWQX23VZ3YF45S2A, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.215823739 (91h47m12.64690387s) [noise=true , address=104.36.82.59:4302 , piece_id=7PQJLFGLE6VC4QFUGZEJX27O6G7JXGTHWPXZJHGGBHEEQNP76VTA, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.216335498 (91h47m12.64741563s) [noise=true , address=104.36.82.19:4405 , piece_id=CQSZNE66AHQEPIZFV3XFJGCSHIYPJOQ4CGYOGQEYMH7RYII73B2Q, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.216784387 (91h47m12.647864518s) [noise=true , address=64.49.13.169:4708 , piece_id=ORFFCDZZWFXYYQINVH6ZR3EDKJDUXLSYDOL3KAEGHWGTM3XYUJWQ, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:18.218082532 (91h47m12.649162663s) [noise=true , address=104.36.82.39:4745 , piece_id=74PNZN5XBA7FNU7GGPF57FCNMEVIARTU7HHZ636RWBZ6GRWODTWQ, seg_pos={0 0}, upload=95648]: Uploading piece... Aug 12 13:13:19.367288725 (91h47m13.798368856s) [upload=95648]: Next returned nil segment Aug 12 13:13:19.367306458 (91h47m13.798386579s) [upload=95648]: Waiting for error group managing segments... Aug 12 13:13:19.684237121 (91h47m14.115317252s) [noise=true , address=64.49.13.140:4509 , piece_id=FMQ46KQDGN3MYI75QL52SJ3OZZZVEYKMDQ3BC5AXSH7DIMIEZC6A, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:19.697492573 (91h47m14.128572704s) [noise=true , address=66.23.193.175:4554 , piece_id=PLZMNRKECIZYURSGDBHNA3QNREEF6XVRHEZMD3JENBHRUWKCOW2A, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:19.715613984 (91h47m14.146694105s) [noise=true , address=64.49.13.169:4708 , piece_id=ORFFCDZZWFXYYQINVH6ZR3EDKJDUXLSYDOL3KAEGHWGTM3XYUJWQ, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.171125142 (91h47m14.602205273s) [noise=true , address=104.36.82.36:4630 , piece_id=V3FZULDW3NWAZVPCO7XOOIZP2O6ZVGB3I3X2IYAJCJBZGTLYXQSA, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.192000363 (91h47m14.623080514s) [noise=true , address=104.36.82.46:4514 , piece_id=7XJNJTDXNNU6WZSXKRNNNVSJSZCGGX7SELNYAPAMYQTOBPA3K6MA, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.200264485 (91h47m14.631344616s) [noise=true , address=104.36.82.62:4414 , piece_id=SVNSZARBEUW5GWUMRZKWXOLZASV2T4FO24HTVULNM3CHMXFMCFWA, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.205494111 (91h47m14.636574242s) [noise=true , address=104.36.82.18:4395 , piece_id=JXPUGWSK35FSUFZCMOO2HOBFFSJ3PZYCYQMI7RFM764DK7ZZYYJQ, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.263317224 (91h47m14.694397355s) [noise=true , address=104.36.82.4:4829 , piece_id=M7AERIUO746K46W6RVIG6D3SYO42KNNO6W7AUVLKPYPGROTFJDKQ, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.308345375 (91h47m14.739425506s) [noise=true , address=104.36.82.58:4267 , piece_id=GFDZW5JPUK7JW2PZVROBX3NZTDW7PCOLCVS2BTSDMNQBAMZOD2GA, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.313057762 (91h47m14.744137893s) [noise=true , address=104.36.82.23:4463 , piece_id=FQ2UL7RYK4R5IMJXW7E4UYOU3XVJZ7TIAL3L2LN4DNJ3NH6K4QCA, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.324555280 (91h47m14.755635411s) [noise=true , address=104.36.82.60:4340 , piece_id=GUJQGU7K3IP3HGQPU4HENOKBJGMN7XXUQ46X2FI5Z47UUBJODQXQ, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.335446939 (91h47m14.76652707s) [noise=true , address=104.36.82.39:4745 , piece_id=74PNZN5XBA7FNU7GGPF57FCNMEVIARTU7HHZ636RWBZ6GRWODTWQ, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.346248966 (91h47m14.777329097s) [noise=true , address=104.36.82.17:4357 , piece_id=VYTJGJ4EAN6SA55WYMHPXA5FNXYDAWHIQTFBSWQX23VZ3YF45S2A, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.359651630 (91h47m14.790731761s) [noise=true , address=104.36.82.19:4405 , piece_id=CQSZNE66AHQEPIZFV3XFJGCSHIYPJOQ4CGYOGQEYMH7RYII73B2Q, seg_pos={0 0}, upload=95648]: Done uploading piece. err: <nil> Aug 12 13:13:20.359674944 (91h47m14.790755075s) [seg_pos={0 0}, upload=95648]: Segment reached optimal threshold of 14 pieces. Aug 12 13:13:20.359804843 (91h47m14.790884974s) [noise=true , address=104.36.82.61:4383 , piece_id=T7LBPLCP5NNOIQ2PJQOAQ37OVL6HZSP6UPLFOCCB3ITFDPI2SN5A, seg_pos={0 0}, upload=95648]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N24q6Li2xavkE61KntNxDPwQzTVHaG4LxyEqFUT1hhGYNQ9mp): protocol: expected piece hash; context canceled; EOF Aug 12 13:13:20.359856541 (91h47m14.790936672s) [noise=true , address=104.36.82.5:4872 , piece_id=TQOMRWQOBESZ67DFENZ5REFSQJ3NL76YQ67SWWVLJT3LIV7L5K6Q, seg_pos={0 0}, upload=95648]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zqxiFpqKeXT9mhXUDuZZmvJkhejC8gcHTQgTadY1xUpQyLyZi): protocol: expected piece hash; context canceled; EOF Aug 12 13:13:20.359872843 (91h47m14.790952964s) [noise=true , address=79.127.201.215:4355 , piece_id=WCGMSZJLQEROU2L4KMY36T47XVQ4GAURFEEVMKAABBBHW5A3LT4A, seg_pos={0 0}, upload=95648]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TkfBtvxa5va3vd1NDoZv1vtyT8FbUPuJJvYyc41FUzXHZJ93D): protocol: expected piece hash; context canceled; EOF Aug 12 13:13:20.359929151 (91h47m14.791009282s) [noise=true , address=104.36.82.59:4302 , piece_id=7PQJLFGLE6VC4QFUGZEJX27O6G7JXGTHWPXZJHGGBHEEQNP76VTA, seg_pos={0 0}, upload=95648]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR): protocol: expected piece hash; context canceled; EOF Aug 12 13:13:20.359947376 (91h47m14.791027507s) [seg_pos={0 0}, upload=95648]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 13:13:20.360005387 (91h47m14.791085518s) [upload=95648]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 13:13:20.360009264 (91h47m14.791089555s) [upload=95648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 13:13:20.680141753 (91h47m15.111221884s) [upload=95648]: Done uploading segments... Aug 12 13:13:20.680160097 (91h47m15.111240228s) [upload=95648]: Upload finished. err: <nil> Duration: 4.059369028s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X...X.....X............................................] duration: 7.815313739s FAIL (2025-08-12 13:13:10 UTC) orbiter warning: context deadline exceeded Duration: 7.815431605s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.......X.........] duration: 6.091295068s FAIL (2025-08-12 13:12:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 6.091324083s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X.X.] duration: 1.860974107s FAIL (2025-08-12 13:12:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after e0afea1286d500709c6eacbc885f14f7096b14f8840e69c9d8166cb8e2debf29 != before 71329f9a8409421df671902efc6f0a67ff5f48d20c74cf8dedf1607d6cf675b6 ======LOGS====== Aug 12 13:11:35.965632559 (91h45m30.39671269s) [upload=95618]: Starting upload Aug 12 13:11:35.965698475 (91h45m30.396778606s) [upload=95618]: Done starting upload Aug 12 13:11:35.965761024 (91h45m30.396841155s) [upload=95618]: Uploading segments... Aug 12 13:11:35.965844103 (91h45m30.396924234s) [upload=95618]: Got next segment. Inline: true Aug 12 13:11:35.965854404 (91h45m30.396934535s) [upload=95618]: Waiting for error group managing segments... Aug 12 13:11:35.965890853 (91h45m30.396970974s) [upload=95618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:11:35.965895442 (91h45m30.396975573s) [upload=95618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 13:11:35.965898257 (91h45m30.396978388s) [upload=95618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 13:11:36.243042387 (91h45m30.674122508s) [upload=95618]: Done uploading segments... Aug 12 13:11:36.243057095 (91h45m30.674137226s) [upload=95618]: Upload finished. err: <nil> Duration:* 1.860998564s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X...X.....X...........................................] duration: 7.197059453s FAIL (2025-08-12 13:12:18 UTC) orbiter warning: context deadline exceeded Duration: 7.197083959s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.......X........] duration: 5.86667068s FAIL (2025-08-12 13:11:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 5.866698263s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X...X.....X..........................................] duration: 8.267418946s FAIL (2025-08-12 13:11:46 UTC) orbiter warning: context deadline exceeded Duration: 8.267437941s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 1.62539216s FAIL (2025-08-12 13:11:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after e0afea1286d500709c6eacbc885f14f7096b14f8840e69c9d8166cb8e2debf29 != before 71329f9a8409421df671902efc6f0a67ff5f48d20c74cf8dedf1607d6cf675b6 ======LOGS====== Aug 12 13:11:35.965632559 (91h45m30.39671269s) [upload=95618]: Starting upload Aug 12 13:11:35.965698475 (91h45m30.396778606s) [upload=95618]: Done starting upload Aug 12 13:11:35.965761024 (91h45m30.396841155s) [upload=95618]: Uploading segments... Aug 12 13:11:35.965844103 (91h45m30.396924234s) [upload=95618]: Got next segment. Inline: true Aug 12 13:11:35.965854404 (91h45m30.396934535s) [upload=95618]: Waiting for error group managing segments... Aug 12 13:11:35.965890853 (91h45m30.396970974s) [upload=95618]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:11:35.965895442 (91h45m30.396975573s) [upload=95618]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 13:11:35.965898257 (91h45m30.396978388s) [upload=95618]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 13:11:36.243042387 (91h45m30.674122508s) [upload=95618]: Done uploading segments... Aug 12 13:11:36.243057095 (91h45m30.674137226s) [upload=95618]: Upload finished. err: <nil> Duration:* 1.625413381s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X...X.....X.........................................] duration: 9.816701598s FAIL (2025-08-12 13:11:04 UTC) orbiter warning: context deadline exceeded Duration: 9.816724221s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.......X.......] duration: 5.858688541s FAIL (2025-08-12 13:11:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 5.858717006s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.942780528s FAIL (2025-08-12 13:10:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 639090bf5d105f33990921aa44bd53776ffe5b17f78e03c0cda546938ca71293 != before a7ce58e3240df692dad8bb0cfe81e99df8a47d2bba9daabc1ff50f255e497039 ======LOGS====== Aug 12 13:09:52.626429887 (91h43m47.057510018s) [upload=95589]: Starting upload Aug 12 13:09:52.626483840 (91h43m47.057563971s) [upload=95589]: Done starting upload Aug 12 13:09:52.626535007 (91h43m47.057615138s) [upload=95589]: Uploading segments... Aug 12 13:09:52.626571638 (91h43m47.057651769s) [upload=95589]: Got next segment. Inline: true Aug 12 13:09:52.626576076 (91h43m47.057656207s) [upload=95589]: Waiting for error group managing segments... Aug 12 13:09:52.626609630 (91h43m47.057689751s) [upload=95589]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:09:52.626613748 (91h43m47.057693879s) [upload=95589]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 13:09:52.626616433 (91h43m47.057696564s) [upload=95589]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 13:09:53.649405382 (91h43m48.080485513s) [upload=95589]: Done uploading segments... Aug 12 13:09:53.649423467 (91h43m48.080503598s) [upload=95589]: Upload finished. err: <nil> Duration:* 1.942805745s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.......X......] duration: 6.25546431s FAIL (2025-08-12 13:10:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 6.255492825s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2.355424667s FAIL (2025-08-12 13:09:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 639090bf5d105f33990921aa44bd53776ffe5b17f78e03c0cda546938ca71293 != before a7ce58e3240df692dad8bb0cfe81e99df8a47d2bba9daabc1ff50f255e497039 ======LOGS====== Aug 12 13:09:52.626429887 (91h43m47.057510018s) [upload=95589]: Starting upload Aug 12 13:09:52.626483840 (91h43m47.057563971s) [upload=95589]: Done starting upload Aug 12 13:09:52.626535007 (91h43m47.057615138s) [upload=95589]: Uploading segments... Aug 12 13:09:52.626571638 (91h43m47.057651769s) [upload=95589]: Got next segment. Inline: true Aug 12 13:09:52.626576076 (91h43m47.057656207s) [upload=95589]: Waiting for error group managing segments... Aug 12 13:09:52.626609630 (91h43m47.057689751s) [upload=95589]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:09:52.626613748 (91h43m47.057693879s) [upload=95589]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 13:09:52.626616433 (91h43m47.057696564s) [upload=95589]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 13:09:53.649405382 (91h43m48.080485513s) [upload=95589]: Done uploading segments... Aug 12 13:09:53.649423467 (91h43m48.080503598s) [upload=95589]: Upload finished. err: <nil> Duration:* 2.355454103s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X...X.....X........................................] duration: 8.987040582s FAIL (2025-08-12 13:09:54 UTC) orbiter warning: context deadline exceeded Duration: 8.987838108s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.901261308s FAIL (2025-08-12 13:09:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.901292618s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X.......X.....] duration: 5.835411235s FAIL (2025-08-12 13:09:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 5.835437446s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X...X.....X.......................................] duration: 8.24931446s FAIL (2025-08-12 13:08:48 UTC) orbiter warning: context deadline exceeded Duration: 8.249338225s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X.......X....] duration: 5.824221529s FAIL (2025-08-12 13:08:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 5.824278366s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.98599469s FAIL (2025-08-12 13:08:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.986023563s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X...X.....X......................................] duration: 7.72499361s FAIL (2025-08-12 13:08:09 UTC) orbiter warning: context deadline exceeded Duration: 7.725019098s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X.......X...] duration: 5.825701531s FAIL (2025-08-12 13:08:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 5.8257227s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.133187952s FAIL (2025-08-12 13:07:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.133218372s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X...X.....X.....................................] duration: 8.166315638s FAIL (2025-08-12 13:07:44 UTC) orbiter warning: context deadline exceeded Duration: 8.166334155s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.095008593s FAIL (2025-08-12 13:07:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.09503302s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X................................................X.......X..] duration: 5.739158687s FAIL (2025-08-12 13:06:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 5.739184919s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X...X.....X....................................] duration: 8.571260341s FAIL (2025-08-12 13:06:43 UTC) orbiter warning: context deadline exceeded Duration: 8.571281441s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.91559384s FAIL (2025-08-12 13:06:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.915623888s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X................................................X.......X.] duration: 5.713740273s FAIL (2025-08-12 13:05:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 5.713762274s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.071901953s FAIL (2025-08-12 13:05:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.071928635s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X...X.....X...................................] duration: 8.067236413s FAIL (2025-08-12 13:05:34 UTC) orbiter warning: context deadline exceeded Duration: 8.067256s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X...X.....X..................................] duration: 7.67512706s FAIL (2025-08-12 13:05:10 UTC) orbiter warning: context deadline exceeded Duration: 7.675148631s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.951275641s FAIL (2025-08-12 13:04:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.951299788s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.X................................................X.......X] duration: 859.880227ms FAIL (2025-08-12 13:04:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 13:04:34.591814755 (91h38m29.022894886s) [upload=95495]: Starting upload Aug 12 13:04:34.591890500 (91h38m29.022970631s) [upload=95495]: Done starting upload Aug 12 13:04:34.592018105 (91h38m29.023098236s) [upload=95495]: Uploading segments... Aug 12 13:04:34.592220582 (91h38m29.023300753s) [upload=95495]: Got next segment. Inline: false Aug 12 13:04:34.592309592 (91h38m29.023389743s) [upload=95495]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 13:04:34.592315855 (91h38m29.023395986s) [upload=95495]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 13:04:34.805360231 (91h38m29.236440372s) [upload=95495]: Next segment err: metaclient: Too Many Requests Aug 12 13:04:34.805397894 (91h38m29.236478075s) [upload=95495]: Done uploading segments... Aug 12 13:04:34.805419025 (91h38m29.236499226s) [upload=95495]: Upload finished. err: metaclient: Too Many Requests Duration: 859.919854ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.072179256s FAIL (2025-08-12 13:04:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.072206668s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X...X.....X.................................] duration: 8.07845206s FAIL (2025-08-12 13:03:58 UTC) orbiter warning: context deadline exceeded Duration: 8.078475383s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.X................................................X.......] duration: 6.437747113s FAIL (2025-08-12 13:03:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 1c4901e29f8282fb8dfb6410397b152d5a3f0352c10f666b53e1ecfea67d9247 != before 63c4e796a4dc0aea04d1e0bc1c359fa7195d17b9551d5a6063c0ecdb8a31fe9a ======LOGS====== Aug 12 12:59:12.800708839 (91h33m7.23178897s) [upload=95402]: Starting upload Aug 12 12:59:12.800745610 (91h33m7.231825731s) [upload=95402]: Done starting upload Aug 12 12:59:12.800854829 (91h33m7.23193496s) [upload=95402]: Uploading segments... Aug 12 12:59:12.800940102 (91h33m7.232020233s) [upload=95402]: Got next segment. Inline: false Aug 12 12:59:12.800972423 (91h33m7.232052554s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:59:12.800976701 (91h33m7.232056832s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 12:59:13.293975951 (91h33m7.725056072s) [seg_pos={0 0}, upload=95402]: Begin upload segment... Aug 12 12:59:13.294009806 (91h33m7.725089937s) [seg_pos={0 0}, upload=95402]: Done begin upload segment. Aug 12 12:59:13.294032800 (91h33m7.725112931s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294091823 (91h33m7.725171954s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294376797 (91h33m7.725456929s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294484213 (91h33m7.725564344s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294708773 (91h33m7.725788904s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294761413 (91h33m7.725841544s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294972638 (91h33m7.726052759s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295033514 (91h33m7.726113645s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295230401 (91h33m7.726310532s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295336333 (91h33m7.726416464s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295499866 (91h33m7.726579997s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296137246 (91h33m7.727217377s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296170229 (91h33m7.72725036s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296530509 (91h33m7.72761064s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296568331 (91h33m7.727648462s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297145084 (91h33m7.728225215s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297221601 (91h33m7.728301722s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.298050016 (91h33m7.729130147s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:14.423541179 (91h33m8.85462131s) [upload=95402]: Next returned nil segment Aug 12 12:59:14.423606575 (91h33m8.854686726s) [upload=95402]: Waiting for error group managing segments... Aug 12 12:59:14.705181615 (91h33m9.136261746s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.735372111 (91h33m9.166452242s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.774871656 (91h33m9.205951788s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.239170281 (91h33m9.670250412s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.250458490 (91h33m9.681538611s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.258971058 (91h33m9.690051189s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.278312825 (91h33m9.709392956s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.279701742 (91h33m9.710781863s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.284405032 (91h33m9.715485163s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.287400984 (91h33m9.718481115s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.323529701 (91h33m9.754609832s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.329353574 (91h33m9.760433695s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.339450693 (91h33m9.770530824s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345335652 (91h33m9.776415783s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345366461 (91h33m9.776446592s) [seg_pos={0 0}, upload=95402]: Segment reached optimal threshold of 14 pieces. Aug 12 12:59:15.345482905 (91h33m9.776563036s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dNdm2kAok8jFgbYE24oXUL9KCJVDDiinLUJgDSYEY3fTEBJec): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345534203 (91h33m9.776614334s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345739886 (91h33m9.776820017s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JzGBbG1khQNggJUXPGbFYejEYjBvYc313aYkiXsj2WDMYCECK): protocol: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection Aug 12 12:59:15.345766617 (91h33m9.776846748s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345783430 (91h33m9.776863561s) [seg_pos={0 0}, upload=95402]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 12:59:15.346057374 (91h33m9.777137505s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 12:59:15.346062734 (91h33m9.777142865s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:59:15.667209619 (91h33m10.098289751s) [upload=95402]: Done uploading segments... Aug 12 12:59:15.667218877 (91h33m10.098299008s) [upload=95402]: Upload finished. err: <nil> Duration: 6.437772262s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.045536709s FAIL (2025-08-12 13:03:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.045570825s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X...X.....X................................] duration: 9.079799164s FAIL (2025-08-12 13:03:13 UTC) orbiter warning: context deadline exceeded Duration: 9.079815114s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.227773983s FAIL (2025-08-12 13:03:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.227807157s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.X................................................X......] duration: 6.013606727s FAIL (2025-08-12 13:02:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 1c4901e29f8282fb8dfb6410397b152d5a3f0352c10f666b53e1ecfea67d9247 != before 63c4e796a4dc0aea04d1e0bc1c359fa7195d17b9551d5a6063c0ecdb8a31fe9a ======LOGS====== Aug 12 12:59:12.800708839 (91h33m7.23178897s) [upload=95402]: Starting upload Aug 12 12:59:12.800745610 (91h33m7.231825731s) [upload=95402]: Done starting upload Aug 12 12:59:12.800854829 (91h33m7.23193496s) [upload=95402]: Uploading segments... Aug 12 12:59:12.800940102 (91h33m7.232020233s) [upload=95402]: Got next segment. Inline: false Aug 12 12:59:12.800972423 (91h33m7.232052554s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:59:12.800976701 (91h33m7.232056832s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 12:59:13.293975951 (91h33m7.725056072s) [seg_pos={0 0}, upload=95402]: Begin upload segment... Aug 12 12:59:13.294009806 (91h33m7.725089937s) [seg_pos={0 0}, upload=95402]: Done begin upload segment. Aug 12 12:59:13.294032800 (91h33m7.725112931s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294091823 (91h33m7.725171954s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294376797 (91h33m7.725456929s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294484213 (91h33m7.725564344s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294708773 (91h33m7.725788904s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294761413 (91h33m7.725841544s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294972638 (91h33m7.726052759s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295033514 (91h33m7.726113645s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295230401 (91h33m7.726310532s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295336333 (91h33m7.726416464s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295499866 (91h33m7.726579997s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296137246 (91h33m7.727217377s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296170229 (91h33m7.72725036s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296530509 (91h33m7.72761064s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296568331 (91h33m7.727648462s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297145084 (91h33m7.728225215s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297221601 (91h33m7.728301722s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.298050016 (91h33m7.729130147s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:14.423541179 (91h33m8.85462131s) [upload=95402]: Next returned nil segment Aug 12 12:59:14.423606575 (91h33m8.854686726s) [upload=95402]: Waiting for error group managing segments... Aug 12 12:59:14.705181615 (91h33m9.136261746s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.735372111 (91h33m9.166452242s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.774871656 (91h33m9.205951788s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.239170281 (91h33m9.670250412s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.250458490 (91h33m9.681538611s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.258971058 (91h33m9.690051189s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.278312825 (91h33m9.709392956s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.279701742 (91h33m9.710781863s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.284405032 (91h33m9.715485163s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.287400984 (91h33m9.718481115s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.323529701 (91h33m9.754609832s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.329353574 (91h33m9.760433695s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.339450693 (91h33m9.770530824s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345335652 (91h33m9.776415783s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345366461 (91h33m9.776446592s) [seg_pos={0 0}, upload=95402]: Segment reached optimal threshold of 14 pieces. Aug 12 12:59:15.345482905 (91h33m9.776563036s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dNdm2kAok8jFgbYE24oXUL9KCJVDDiinLUJgDSYEY3fTEBJec): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345534203 (91h33m9.776614334s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345739886 (91h33m9.776820017s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JzGBbG1khQNggJUXPGbFYejEYjBvYc313aYkiXsj2WDMYCECK): protocol: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection Aug 12 12:59:15.345766617 (91h33m9.776846748s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345783430 (91h33m9.776863561s) [seg_pos={0 0}, upload=95402]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 12:59:15.346057374 (91h33m9.777137505s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 12:59:15.346062734 (91h33m9.777142865s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:59:15.667209619 (91h33m10.098289751s) [upload=95402]: Done uploading segments... Aug 12 12:59:15.667218877 (91h33m10.098299008s) [upload=95402]: Upload finished. err: <nil> Duration: 6.013621496s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.335447144s FAIL (2025-08-12 13:02:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.335489746s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.X................................................X.....] duration: 6.0922572s FAIL (2025-08-12 13:02:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 1c4901e29f8282fb8dfb6410397b152d5a3f0352c10f666b53e1ecfea67d9247 != before 63c4e796a4dc0aea04d1e0bc1c359fa7195d17b9551d5a6063c0ecdb8a31fe9a ======LOGS====== Aug 12 12:59:12.800708839 (91h33m7.23178897s) [upload=95402]: Starting upload Aug 12 12:59:12.800745610 (91h33m7.231825731s) [upload=95402]: Done starting upload Aug 12 12:59:12.800854829 (91h33m7.23193496s) [upload=95402]: Uploading segments... Aug 12 12:59:12.800940102 (91h33m7.232020233s) [upload=95402]: Got next segment. Inline: false Aug 12 12:59:12.800972423 (91h33m7.232052554s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:59:12.800976701 (91h33m7.232056832s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 12:59:13.293975951 (91h33m7.725056072s) [seg_pos={0 0}, upload=95402]: Begin upload segment... Aug 12 12:59:13.294009806 (91h33m7.725089937s) [seg_pos={0 0}, upload=95402]: Done begin upload segment. Aug 12 12:59:13.294032800 (91h33m7.725112931s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294091823 (91h33m7.725171954s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294376797 (91h33m7.725456929s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294484213 (91h33m7.725564344s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294708773 (91h33m7.725788904s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294761413 (91h33m7.725841544s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294972638 (91h33m7.726052759s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295033514 (91h33m7.726113645s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295230401 (91h33m7.726310532s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295336333 (91h33m7.726416464s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295499866 (91h33m7.726579997s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296137246 (91h33m7.727217377s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296170229 (91h33m7.72725036s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296530509 (91h33m7.72761064s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296568331 (91h33m7.727648462s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297145084 (91h33m7.728225215s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297221601 (91h33m7.728301722s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.298050016 (91h33m7.729130147s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:14.423541179 (91h33m8.85462131s) [upload=95402]: Next returned nil segment Aug 12 12:59:14.423606575 (91h33m8.854686726s) [upload=95402]: Waiting for error group managing segments... Aug 12 12:59:14.705181615 (91h33m9.136261746s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.735372111 (91h33m9.166452242s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.774871656 (91h33m9.205951788s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.239170281 (91h33m9.670250412s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.250458490 (91h33m9.681538611s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.258971058 (91h33m9.690051189s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.278312825 (91h33m9.709392956s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.279701742 (91h33m9.710781863s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.284405032 (91h33m9.715485163s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.287400984 (91h33m9.718481115s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.323529701 (91h33m9.754609832s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.329353574 (91h33m9.760433695s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.339450693 (91h33m9.770530824s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345335652 (91h33m9.776415783s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345366461 (91h33m9.776446592s) [seg_pos={0 0}, upload=95402]: Segment reached optimal threshold of 14 pieces. Aug 12 12:59:15.345482905 (91h33m9.776563036s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dNdm2kAok8jFgbYE24oXUL9KCJVDDiinLUJgDSYEY3fTEBJec): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345534203 (91h33m9.776614334s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345739886 (91h33m9.776820017s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JzGBbG1khQNggJUXPGbFYejEYjBvYc313aYkiXsj2WDMYCECK): protocol: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection Aug 12 12:59:15.345766617 (91h33m9.776846748s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345783430 (91h33m9.776863561s) [seg_pos={0 0}, upload=95402]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 12:59:15.346057374 (91h33m9.777137505s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 12:59:15.346062734 (91h33m9.777142865s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:59:15.667209619 (91h33m10.098289751s) [upload=95402]: Done uploading segments... Aug 12 12:59:15.667218877 (91h33m10.098299008s) [upload=95402]: Upload finished. err: <nil> Duration: 6.092279041s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X...X.....X...............................] duration: 7.615525876s FAIL (2025-08-12 13:02:06 UTC) orbiter warning: context deadline exceeded Duration: 7.615547757s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.122613392s FAIL (2025-08-12 13:02:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 12:27:59.013658005 (91h1m53.444738126s) [upload=94840]: Starting upload Aug 12 12:27:59.013706879 (91h1m53.44478701s) [upload=94840]: Done starting upload Aug 12 12:27:59.013819735 (91h1m53.444899866s) [upload=94840]: Uploading segments... Aug 12 12:27:59.013930837 (91h1m53.445010968s) [upload=94840]: Got next segment. Inline: true Aug 12 12:27:59.013937610 (91h1m53.445017741s) [upload=94840]: Waiting for error group managing segments... Aug 12 12:27:59.013987075 (91h1m53.445067206s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:27:59.014003507 (91h1m53.445083638s) [upload=94840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 12:27:59.014007474 (91h1m53.445087615s) [upload=94840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:27:59.231630303 (91h1m53.662710434s) [upload=94840]: Done uploading segments... Aug 12 12:27:59.231647326 (91h1m53.662727457s) [upload=94840]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.122638259s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.X................................................X....] duration: 5.725306879s FAIL (2025-08-12 13:01:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 1c4901e29f8282fb8dfb6410397b152d5a3f0352c10f666b53e1ecfea67d9247 != before 63c4e796a4dc0aea04d1e0bc1c359fa7195d17b9551d5a6063c0ecdb8a31fe9a ======LOGS====== Aug 12 12:59:12.800708839 (91h33m7.23178897s) [upload=95402]: Starting upload Aug 12 12:59:12.800745610 (91h33m7.231825731s) [upload=95402]: Done starting upload Aug 12 12:59:12.800854829 (91h33m7.23193496s) [upload=95402]: Uploading segments... Aug 12 12:59:12.800940102 (91h33m7.232020233s) [upload=95402]: Got next segment. Inline: false Aug 12 12:59:12.800972423 (91h33m7.232052554s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:59:12.800976701 (91h33m7.232056832s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 12:59:13.293975951 (91h33m7.725056072s) [seg_pos={0 0}, upload=95402]: Begin upload segment... Aug 12 12:59:13.294009806 (91h33m7.725089937s) [seg_pos={0 0}, upload=95402]: Done begin upload segment. Aug 12 12:59:13.294032800 (91h33m7.725112931s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294091823 (91h33m7.725171954s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294376797 (91h33m7.725456929s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294484213 (91h33m7.725564344s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294708773 (91h33m7.725788904s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294761413 (91h33m7.725841544s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294972638 (91h33m7.726052759s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295033514 (91h33m7.726113645s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295230401 (91h33m7.726310532s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295336333 (91h33m7.726416464s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295499866 (91h33m7.726579997s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296137246 (91h33m7.727217377s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296170229 (91h33m7.72725036s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296530509 (91h33m7.72761064s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296568331 (91h33m7.727648462s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297145084 (91h33m7.728225215s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297221601 (91h33m7.728301722s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.298050016 (91h33m7.729130147s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:14.423541179 (91h33m8.85462131s) [upload=95402]: Next returned nil segment Aug 12 12:59:14.423606575 (91h33m8.854686726s) [upload=95402]: Waiting for error group managing segments... Aug 12 12:59:14.705181615 (91h33m9.136261746s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.735372111 (91h33m9.166452242s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.774871656 (91h33m9.205951788s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.239170281 (91h33m9.670250412s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.250458490 (91h33m9.681538611s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.258971058 (91h33m9.690051189s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.278312825 (91h33m9.709392956s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.279701742 (91h33m9.710781863s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.284405032 (91h33m9.715485163s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.287400984 (91h33m9.718481115s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.323529701 (91h33m9.754609832s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.329353574 (91h33m9.760433695s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.339450693 (91h33m9.770530824s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345335652 (91h33m9.776415783s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345366461 (91h33m9.776446592s) [seg_pos={0 0}, upload=95402]: Segment reached optimal threshold of 14 pieces. Aug 12 12:59:15.345482905 (91h33m9.776563036s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dNdm2kAok8jFgbYE24oXUL9KCJVDDiinLUJgDSYEY3fTEBJec): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345534203 (91h33m9.776614334s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345739886 (91h33m9.776820017s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JzGBbG1khQNggJUXPGbFYejEYjBvYc313aYkiXsj2WDMYCECK): protocol: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection Aug 12 12:59:15.345766617 (91h33m9.776846748s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345783430 (91h33m9.776863561s) [seg_pos={0 0}, upload=95402]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 12:59:15.346057374 (91h33m9.777137505s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 12:59:15.346062734 (91h33m9.777142865s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:59:15.667209619 (91h33m10.098289751s) [upload=95402]: Done uploading segments... Aug 12 12:59:15.667218877 (91h33m10.098299008s) [upload=95402]: Upload finished. err: <nil> Duration: 5.725333069s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X....X.X................................................X...] duration: 6.675153566s FAIL (2025-08-12 13:01:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 1c4901e29f8282fb8dfb6410397b152d5a3f0352c10f666b53e1ecfea67d9247 != before 63c4e796a4dc0aea04d1e0bc1c359fa7195d17b9551d5a6063c0ecdb8a31fe9a ======LOGS====== Aug 12 12:59:12.800708839 (91h33m7.23178897s) [upload=95402]: Starting upload Aug 12 12:59:12.800745610 (91h33m7.231825731s) [upload=95402]: Done starting upload Aug 12 12:59:12.800854829 (91h33m7.23193496s) [upload=95402]: Uploading segments... Aug 12 12:59:12.800940102 (91h33m7.232020233s) [upload=95402]: Got next segment. Inline: false Aug 12 12:59:12.800972423 (91h33m7.232052554s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:59:12.800976701 (91h33m7.232056832s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 12:59:13.293975951 (91h33m7.725056072s) [seg_pos={0 0}, upload=95402]: Begin upload segment... Aug 12 12:59:13.294009806 (91h33m7.725089937s) [seg_pos={0 0}, upload=95402]: Done begin upload segment. Aug 12 12:59:13.294032800 (91h33m7.725112931s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294091823 (91h33m7.725171954s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294376797 (91h33m7.725456929s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294484213 (91h33m7.725564344s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294708773 (91h33m7.725788904s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294761413 (91h33m7.725841544s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294972638 (91h33m7.726052759s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295033514 (91h33m7.726113645s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295230401 (91h33m7.726310532s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295336333 (91h33m7.726416464s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295499866 (91h33m7.726579997s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296137246 (91h33m7.727217377s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296170229 (91h33m7.72725036s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296530509 (91h33m7.72761064s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296568331 (91h33m7.727648462s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297145084 (91h33m7.728225215s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297221601 (91h33m7.728301722s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.298050016 (91h33m7.729130147s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:14.423541179 (91h33m8.85462131s) [upload=95402]: Next returned nil segment Aug 12 12:59:14.423606575 (91h33m8.854686726s) [upload=95402]: Waiting for error group managing segments... Aug 12 12:59:14.705181615 (91h33m9.136261746s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.735372111 (91h33m9.166452242s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.774871656 (91h33m9.205951788s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.239170281 (91h33m9.670250412s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.250458490 (91h33m9.681538611s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.258971058 (91h33m9.690051189s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.278312825 (91h33m9.709392956s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.279701742 (91h33m9.710781863s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.284405032 (91h33m9.715485163s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.287400984 (91h33m9.718481115s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.323529701 (91h33m9.754609832s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.329353574 (91h33m9.760433695s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.339450693 (91h33m9.770530824s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345335652 (91h33m9.776415783s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345366461 (91h33m9.776446592s) [seg_pos={0 0}, upload=95402]: Segment reached optimal threshold of 14 pieces. Aug 12 12:59:15.345482905 (91h33m9.776563036s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dNdm2kAok8jFgbYE24oXUL9KCJVDDiinLUJgDSYEY3fTEBJec): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345534203 (91h33m9.776614334s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345739886 (91h33m9.776820017s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JzGBbG1khQNggJUXPGbFYejEYjBvYc313aYkiXsj2WDMYCECK): protocol: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection Aug 12 12:59:15.345766617 (91h33m9.776846748s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345783430 (91h33m9.776863561s) [seg_pos={0 0}, upload=95402]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 12:59:15.346057374 (91h33m9.777137505s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 12:59:15.346062734 (91h33m9.777142865s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:59:15.667209619 (91h33m10.098289751s) [upload=95402]: Done uploading segments... Aug 12 12:59:15.667218877 (91h33m10.098299008s) [upload=95402]: Upload finished. err: <nil> Duration: 6.675181259s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X...X.....X..............................] duration: 7.84306112s FAIL (2025-08-12 13:01:24 UTC) orbiter warning: context deadline exceeded Duration: 7.843086469s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X....X.X................................................X..] duration: 6.248964819s FAIL (2025-08-12 13:00:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 1c4901e29f8282fb8dfb6410397b152d5a3f0352c10f666b53e1ecfea67d9247 != before 63c4e796a4dc0aea04d1e0bc1c359fa7195d17b9551d5a6063c0ecdb8a31fe9a ======LOGS====== Aug 12 12:59:12.800708839 (91h33m7.23178897s) [upload=95402]: Starting upload Aug 12 12:59:12.800745610 (91h33m7.231825731s) [upload=95402]: Done starting upload Aug 12 12:59:12.800854829 (91h33m7.23193496s) [upload=95402]: Uploading segments... Aug 12 12:59:12.800940102 (91h33m7.232020233s) [upload=95402]: Got next segment. Inline: false Aug 12 12:59:12.800972423 (91h33m7.232052554s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 12:59:12.800976701 (91h33m7.232056832s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 12:59:13.293975951 (91h33m7.725056072s) [seg_pos={0 0}, upload=95402]: Begin upload segment... Aug 12 12:59:13.294009806 (91h33m7.725089937s) [seg_pos={0 0}, upload=95402]: Done begin upload segment. Aug 12 12:59:13.294032800 (91h33m7.725112931s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294091823 (91h33m7.725171954s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294376797 (91h33m7.725456929s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294484213 (91h33m7.725564344s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294708773 (91h33m7.725788904s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294761413 (91h33m7.725841544s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.294972638 (91h33m7.726052759s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295033514 (91h33m7.726113645s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295230401 (91h33m7.726310532s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295336333 (91h33m7.726416464s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.295499866 (91h33m7.726579997s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296137246 (91h33m7.727217377s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296170229 (91h33m7.72725036s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296530509 (91h33m7.72761064s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.296568331 (91h33m7.727648462s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297145084 (91h33m7.728225215s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.297221601 (91h33m7.728301722s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:13.298050016 (91h33m7.729130147s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Uploading piece... Aug 12 12:59:14.423541179 (91h33m8.85462131s) [upload=95402]: Next returned nil segment Aug 12 12:59:14.423606575 (91h33m8.854686726s) [upload=95402]: Waiting for error group managing segments... Aug 12 12:59:14.705181615 (91h33m9.136261746s) [noise=true , address=64.49.13.139:4473 , piece_id=V2OITITY5EPORRQ2FHDYGHUHYAGMLRE5FAA3SZA3YG7CEJQSOODQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.735372111 (91h33m9.166452242s) [noise=true , address=64.49.13.140:4516 , piece_id=NVRGR4PMJ2G3JO62CJNMS4ZT4AN66AMYMT4TQDIDJQ3D4PPAAZAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:14.774871656 (91h33m9.205951788s) [noise=true , address=64.49.13.170:4719 , piece_id=7QG6ARKOMUVRHGYBNISESOECGX2KE7OMHLEJCJS3OLU55RHZMJDA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.239170281 (91h33m9.670250412s) [noise=true , address=77.90.31.15:4702 , piece_id=DCZ5A44E2DZ5QQINWOGGDO6FBKV2NISKJKXUZZ6JJDR6YAOARLXA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.250458490 (91h33m9.681538611s) [noise=true , address=104.36.82.37:4673 , piece_id=64LLB2EAAFQDPDKL7K7RHMW7FVAMMG6NLD53DVIMIHM4Z2BLKYPQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.258971058 (91h33m9.690051189s) [noise=true , address=104.36.82.61:4381 , piece_id=UTZJKYN526MP4FUIKA3KWXBMDUDN4BWAIBKICVOZHAUEIEOR3UAA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.278312825 (91h33m9.709392956s) [noise=true , address=104.36.82.59:4330 , piece_id=VELJ7BDNYDTKPC4MD4UI7OFS3D4BYYLH5TIIK3EOMVJ4GNIX577Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.279701742 (91h33m9.710781863s) [noise=true , address=104.36.82.38:4692 , piece_id=54VG6ND4OVLY7DQIRGEUEITUPF2F6VR53ERGNPDLHMKWR3YM63AQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.284405032 (91h33m9.715485163s) [noise=true , address=104.36.82.23:4474 , piece_id=RLSGSRF54ENKBBEYQA2GRRJNUWUPYR33NZUE52ZMNQ6OGJDUDN7Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.287400984 (91h33m9.718481115s) [noise=true , address=104.36.82.8:4976 , piece_id=2QZEPB2D424DA7Y72WCHOAE4THE5Q666KTS737G3RWGWA7GPPMRA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.323529701 (91h33m9.754609832s) [noise=true , address=104.36.82.35:4597 , piece_id=VVBGN2CCLVPA2LFELEBQFQQ6OKTZ43OOUAEMYBKDTTF6LEI5BRIA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.329353574 (91h33m9.760433695s) [noise=true , address=104.36.82.60:4365 , piece_id=TWREF4DLPBYJE333ZWQ4CMQLZW7FZ7BY4BQYDRBQ5DOKJAQ2TCUQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.339450693 (91h33m9.770530824s) [noise=true , address=104.36.82.58:4296 , piece_id=YPUA4CIC2KUXQ5JOQEUN32LIXXSCON4XUDRIOGLFD6I276LXC57A, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345335652 (91h33m9.776415783s) [noise=true , address=104.36.82.17:4332 , piece_id=LJMRJJ2CP3AKDGO6MKCU42H63TFVNFS6E36PQ4G7MEAIRXDDCM3Q, seg_pos={0 0}, upload=95402]: Done uploading piece. err: <nil> Aug 12 12:59:15.345366461 (91h33m9.776446592s) [seg_pos={0 0}, upload=95402]: Segment reached optimal threshold of 14 pieces. Aug 12 12:59:15.345482905 (91h33m9.776563036s) [noise=true , address=104.36.82.44:4872 , piece_id=3FTYGX3434K2MXD4O2UL5EZX2TSDLS4CNCOUG7RPOR2DU7Q2F4BA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dNdm2kAok8jFgbYE24oXUL9KCJVDDiinLUJgDSYEY3fTEBJec): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345534203 (91h33m9.776614334s) [noise=true , address=104.36.82.34:4556 , piece_id=RE4ZQEDM2BTUUY4V6QXC2QB5Q5TDVGONY3AW7POE5BPVXJBTG4XA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i62GRFASf7UPVGhUofmRkz3zZSyhq4fDycgyYtfD7pHR3w91v): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345739886 (91h33m9.776820017s) [noise=true , address=104.36.82.62:4432 , piece_id=WIP3EVDN2MXULLAGVYGCBV3WZS72H6N74HLPKQGY5NLINZWLOCNQ, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JzGBbG1khQNggJUXPGbFYejEYjBvYc313aYkiXsj2WDMYCECK): protocol: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50478->104.36.82.62:4432: use of closed network connection Aug 12 12:59:15.345766617 (91h33m9.776846748s) [noise=true , address=79.127.201.215:4358 , piece_id=57Q7ARX62UBLOXB6H7MAP2KDCTDXLQP4F2CTQMW7P2BZQZLHHROA, seg_pos={0 0}, upload=95402]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cy2na9tb5e6cEXSiM9E4yuZszRDNv4gQU7VBUXnQCSRoPHTA6): protocol: expected piece hash; context canceled; EOF Aug 12 12:59:15.345783430 (91h33m9.776863561s) [seg_pos={0 0}, upload=95402]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 12:59:15.346057374 (91h33m9.777137505s) [upload=95402]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 12:59:15.346062734 (91h33m9.777142865s) [upload=95402]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 12:59:15.667209619 (91h33m10.098289751s) [upload=95402]: Done uploading segments... Aug 12 12:59:15.667218877 (91h33m10.098299008s) [upload=95402]: Upload finished. err: <nil> Duration: 6.248993163s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)