Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X.....................X...............................] duration: 1.934465153s FAIL (2025-08-28 03:40:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.934497005s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 27.615209951s FAIL (2025-08-28 03:40:42 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 27.615238998s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............XX....X.......................................] duration: 1.534958053s FAIL (2025-08-28 03:40:29 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.534983834s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.954257527s FAIL (2025-08-28 03:40:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.954282015s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X.....................X..............................] duration: 1.872738044s FAIL (2025-08-28 03:40:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.872763873s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.758740343s FAIL (2025-08-28 03:39:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.758764119s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.648157956s FAIL (2025-08-28 03:39:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.648202681s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............XX....X......................................] duration: 1.412340938s FAIL (2025-08-28 03:39:29 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.412364803s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 5.049811758s FAIL (2025-08-28 03:39:19 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.04983397s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [................XX....X.....................................] duration: 22.152834122s FAIL (2025-08-28 03:38:58 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 22.152859893s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X.....................X.............................] duration: 1.977227689s FAIL (2025-08-28 03:38:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.977255452s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 30.00371164s FAIL (2025-08-28 03:38:51 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 30.00373278s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.............X.............................................] duration: 5.715895226s FAIL (2025-08-28 03:38:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.715924141s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X.....................X............................] duration: 2.128266954s FAIL (2025-08-28 03:38:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.128295909s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 5.127548621s FAIL (2025-08-28 03:38:05 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.127577717s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................XX....X....................................] duration: 16.118132184s FAIL (2025-08-28 03:37:52 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 16.118172371s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X.....................X...........................] duration: 2.006308989s FAIL (2025-08-28 03:37:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.006348845s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.............X............................................] duration: 5.6024031s FAIL (2025-08-28 03:37:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.602431113s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................XX....X...................................] duration: 1.471546202s FAIL (2025-08-28 03:37:09 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.471594154s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 4.955993385s FAIL (2025-08-28 03:36:51 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 4.956010638s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 30.00347699s FAIL (2025-08-28 03:36:46 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 30.003501357s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.............X...........................................] duration: 5.658295233s FAIL (2025-08-28 03:36:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.658322094s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X.....................X..........................] duration: 1.916498163s FAIL (2025-08-28 03:36:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.916522461s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X.....................X.........................] duration: 1.907125222s FAIL (2025-08-28 03:36:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.907152423s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 27.098968236s FAIL (2025-08-28 03:36:11 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 27.099000258s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X.............X..........................................] duration: 5.774697577s FAIL (2025-08-28 03:36:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.774728256s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................XX....X..................................] duration: 1.344106669s FAIL (2025-08-28 03:35:56 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.344127698s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X.............X.........................................] duration: 5.797457045s FAIL (2025-08-28 03:35:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.797486173s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 22.670487102s FAIL (2025-08-28 03:35:25 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 22.670508132s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X.....................X........................] duration: 2.084455545s FAIL (2025-08-28 03:35:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.084482907s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................XX....X.................................] duration: 1.410289152s FAIL (2025-08-28 03:35:18 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.410310241s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................XX....X................................] duration: 1.367070589s FAIL (2025-08-28 03:34:58 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.367099064s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 23.516920538s FAIL (2025-08-28 03:34:52 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 23.516948462s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X.............X........................................] duration: 5.907020794s FAIL (2025-08-28 03:34:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.907050893s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................XX....X...............................] duration: 1.476232545s FAIL (2025-08-28 03:34:23 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.476259786s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X.....................X.......................] duration: 2.038051185s FAIL (2025-08-28 03:34:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.038074009s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................XX....X..............................] duration: 1.427977638s FAIL (2025-08-28 03:34:04 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.428011783s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 4.899303154s FAIL (2025-08-28 03:33:59 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 4.899339624s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.............X.......................................] duration: 5.75429076s FAIL (2025-08-28 03:33:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.754319507s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................XX....X.............................] duration: 28.944416422s FAIL (2025-08-28 03:33:46 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 28.944445639s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X.............X......................................] duration: 5.79549434s FAIL (2025-08-28 03:33:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.795522004s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X.....................X......................] duration: 2.040035792s FAIL (2025-08-28 03:33:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.04006565s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 4.902768635s FAIL (2025-08-28 03:33:22 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 4.902796169s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................XX....X............................] duration: 1.559308977s FAIL (2025-08-28 03:32:48 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.559340418s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X.............X.....................................] duration: 5.863812198s FAIL (2025-08-28 03:32:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e209a8d13687cf7a6f02df6b45d8d12c524c85e633e9df56a30516eb5087b9bf != before 60324e0e8d7f54d8981dc3b988ba817dd02f561f902eb2a587ac34c2bd25ef40 ======LOGS====== Aug 28 03:05:47.763806534 (9h27m26.926407503s) [upload=9892]: Starting upload Aug 28 03:05:47.763911796 (9h27m26.926512734s) [upload=9892]: Done starting upload Aug 28 03:05:47.764016726 (9h27m26.926617724s) [upload=9892]: Uploading segments... Aug 28 03:05:47.764358060 (9h27m26.926959038s) [upload=9892]: Got next segment. Inline: false Aug 28 03:05:47.764422422 (9h27m26.927023411s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:05:47.764431030 (9h27m26.927032008s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 28 03:05:48.234221246 (9h27m27.396822184s) [seg_pos={0 0}, upload=9892]: Begin upload segment... Aug 28 03:05:48.234259661 (9h27m27.396860589s) [seg_pos={0 0}, upload=9892]: Done begin upload segment. Aug 28 03:05:48.234284498 (9h27m27.396885436s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234673122 (9h27m27.39727406s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.234960281 (9h27m27.397561219s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235236549 (9h27m27.397837487s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235508530 (9h27m27.398109468s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235738500 (9h27m27.398339438s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.235962919 (9h27m27.398563847s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236175156 (9h27m27.398776094s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236393464 (9h27m27.398994402s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236580681 (9h27m27.399181619s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.236811142 (9h27m27.399412081s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237001377 (9h27m27.399602315s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237201249 (9h27m27.399802187s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237392154 (9h27m27.399993092s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237580695 (9h27m27.400181633s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237768554 (9h27m27.400369493s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.237992633 (9h27m27.400593561s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:48.238284462 (9h27m27.4008854s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Uploading piece... Aug 28 03:05:49.387031227 (9h27m28.549632165s) [upload=9892]: Next returned nil segment Aug 28 03:05:49.387133704 (9h27m28.549734652s) [upload=9892]: Waiting for error group managing segments... Aug 28 03:05:49.675680084 (9h27m28.838281022s) [noise=true , address=79.127.205.234:4610 , piece_id=TMSWYHRYL2BSUNAFLDOGUDCESCY3R5XFLSFUMS45QTTEF7W37E6A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.760536010 (9h27m28.923136938s) [noise=true , address=64.49.13.140:4508 , piece_id=IZF7CQVMQ4YEJ2IZ4UIVKPMYBB5EDG6KGWVV2LQHJVTVFGY6ABTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:49.801388398 (9h27m28.963989336s) [noise=true , address=79.127.205.246:4278 , piece_id=LLRUSFY7JDFYJX4SV6GWQWFM57D4A73PBMPX32FFRMNYNJNGVPIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.123396896 (9h27m29.285997834s) [noise=true , address=77.90.31.11:4574 , piece_id=V36GJKMBWGXOWA7QADHC3AXAXIG6E46EGXL6SZTGNFVMGQH75IYQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.127851609 (9h27m29.290452557s) [noise=true , address=45.91.116.13:4170 , piece_id=S3OHCRDT7XQ6IE6W43XKOQQ3ZTJA4A4XOBQ7ZMIWLGRO3VZ3AH5A, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.129364594 (9h27m29.291965532s) [noise=true , address=77.90.31.21:4885 , piece_id=XORKOHU2M3X27AOS3MZFIKIWNR2G4MB6GM5EKN4GMYR4W3KB2UUQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.138208316 (9h27m29.300809254s) [noise=true , address=77.90.31.15:4705 , piece_id=BKTQWN4U54L2T4CIUBYJMHAFDIWQTW6ECYY766DCV4JT7YXRRMIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.155487328 (9h27m29.318088267s) [noise=true , address=104.36.82.7:4915 , piece_id=4QXJ3KDOS7D43X5OSDQ5MJ6OD35CRMHRK3CZHHMDP54TIPALNUTQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.182833093 (9h27m29.345434031s) [noise=true , address=45.91.116.8:4075 , piece_id=SZUTHYVPBBDE7W66K2Y7FRFUDH6SINV34FCQ7DLCA2GEVG52GSPQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.219298215 (9h27m29.381899153s) [noise=true , address=77.90.31.16:4759 , piece_id=7MUAUANXODKE76FN5HDU6ZYZHTFRW2KTRNPGDNCNXYGHLM4OUZZA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.222611274 (9h27m29.385212213s) [noise=true , address=45.91.116.4:4941 , piece_id=5LNSDYMGEZZYUNHXUXXJFKGJV62H2WRETRCQZV6FKVMDTTT4GE7Q, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.225730693 (9h27m29.388331631s) [noise=true , address=104.36.82.5:4845 , piece_id=XYVLKRQOBYQAKCOZ6DST335JCO54GYLV3DHD6DEPUWDYCAEYLVAQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.270659441 (9h27m29.433260379s) [noise=true , address=45.91.116.10:4108 , piece_id=GVLYMNEHUGG5GA2HSN2D5EUC5EQRPK2ZICVOANU2UOQLQ5YJ2EVA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285768922 (9h27m29.44836986s) [noise=true , address=77.90.31.14:4687 , piece_id=YD3WGCQFT2DDN6BMGSXOHB7MGXQ5LAV25U5ZPBDYGI7S4O3ZWKIA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: <nil> Aug 28 03:05:50.285794541 (9h27m29.448395479s) [seg_pos={0 0}, upload=9892]: Segment reached optimal threshold of 14 pieces. Aug 28 03:05:50.285940640 (9h27m29.448541578s) [noise=true , address=77.90.31.18:4830 , piece_id=GELPZ3SG4GES6GDLNOMLS2X7SBE45Z74MTSUW42LOFS7TEZRK6MQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286069848 (9h27m29.448670786s) [noise=true , address=104.36.82.60:4357 , piece_id=DI6KFZQCETLYC576LMZDZ6WEF5XV7XDRLVI7TTLHPNP72DVO3BIQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EmBobdmk16xm9girzx3asYwkhdCKKZCCPrBLwWTS4ujMdGmi): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286132117 (9h27m29.448733045s) [noise=true , address=104.36.82.52:4119 , piece_id=63YHOA45JLQCZOHNDN4URJB4CCJA4XDHL2E42BQRKIUETMNNQCTA, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mTLjL6Y6VVjqztwVXYif4v7tyftjLafztLUibajksMHCFraFG): protocol: expected piece hash; context canceled; EOF Aug 28 03:05:50.286167564 (9h27m29.448768492s) [noise=true , address=104.36.82.5:4854 , piece_id=D2FHWII2PRLTGMM24BUOWR55PQLGP3LB6RA2GRLY64M3VPBY37WQ, seg_pos={0 0}, upload=9892]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy): protocol: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41190->104.36.82.5:4854: use of closed network connection Aug 28 03:05:50.286188635 (9h27m29.448789573s) [seg_pos={0 0}, upload=9892]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 28 03:05:50.286251655 (9h27m29.448852593s) [upload=9892]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 28 03:05:50.286255904 (9h27m29.448856842s) [upload=9892]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:05:50.596989558 (9h27m29.759590496s) [upload=9892]: Done uploading segments... Aug 28 03:05:50.597023813 (9h27m29.759624741s) [upload=9892]: Upload finished. err: <nil> Duration: 5.863888603s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X.....................X.....................] duration: 2.276063598s FAIL (2025-08-28 03:32:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.27617478s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 4.972392763s FAIL (2025-08-28 03:32:29 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 4.972422791s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X................X.....................X....................] duration: 2.042787988s FAIL (2025-08-28 03:32:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.042805232s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........................XX....X...........................] duration: 1.454486821s FAIL (2025-08-28 03:32:10 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.454529733s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X................X.....................X...................] duration: 2.185457983s FAIL (2025-08-28 03:31:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 03:18:05.467703419 (9h39m44.630304377s) [upload=10114]: Starting upload Aug 28 03:18:05.467798450 (9h39m44.630399418s) [upload=10114]: Done starting upload Aug 28 03:18:05.467888944 (9h39m44.630489932s) [upload=10114]: Uploading segments... Aug 28 03:18:05.467959088 (9h39m44.630560066s) [upload=10114]: Got next segment. Inline: true Aug 28 03:18:05.467967052 (9h39m44.630568031s) [upload=10114]: Waiting for error group managing segments... Aug 28 03:18:05.468027860 (9h39m44.630628848s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 28 03:18:05.468035835 (9h39m44.630636813s) [upload=10114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 28 03:18:05.468041766 (9h39m44.630642754s) [upload=10114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 28 03:18:05.691590395 (9h39m44.854191333s) [upload=10114]: Done uploading segments... Aug 28 03:18:05.691608059 (9h39m44.854208997s) [upload=10114]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.185478363s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)