Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................X..........X........................X.....] duration: 2.027412027s FAIL (2025-08-09 07:39:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 07:35:17.201671304 (14h9m11.632751425s) [upload=14777]: Starting upload Aug 9 07:35:17.201718444 (14h9m11.632798575s) [upload=14777]: Done starting upload Aug 9 07:35:17.201821532 (14h9m11.632901663s) [upload=14777]: Uploading segments... Aug 9 07:35:17.201864483 (14h9m11.632944614s) [upload=14777]: Got next segment. Inline: true Aug 9 07:35:17.201870205 (14h9m11.632950336s) [upload=14777]: Waiting for error group managing segments... Aug 9 07:35:17.201904821 (14h9m11.632984952s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:35:17.201909380 (14h9m11.632989511s) [upload=14777]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:35:17.201911984 (14h9m11.632992116s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:35:17.417157919 (14h9m11.84823805s) [upload=14777]: Done uploading segments... Aug 9 07:35:17.417175132 (14h9m11.848255263s) [upload=14777]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.02744061s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............X..X..X......X.......X.........................] duration: 5.951912011s FAIL (2025-08-09 07:38:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.951934425s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X....................X........................X.........] duration: 5.606029766s FAIL (2025-08-09 07:38:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.606082126s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............X..X..X......X.......X........................] duration: 6.057733321s FAIL (2025-08-09 07:38:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.057755993s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................X..........X........................X....] duration: 1.86503243s FAIL (2025-08-09 07:38:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 07:35:17.201671304 (14h9m11.632751425s) [upload=14777]: Starting upload Aug 9 07:35:17.201718444 (14h9m11.632798575s) [upload=14777]: Done starting upload Aug 9 07:35:17.201821532 (14h9m11.632901663s) [upload=14777]: Uploading segments... Aug 9 07:35:17.201864483 (14h9m11.632944614s) [upload=14777]: Got next segment. Inline: true Aug 9 07:35:17.201870205 (14h9m11.632950336s) [upload=14777]: Waiting for error group managing segments... Aug 9 07:35:17.201904821 (14h9m11.632984952s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:35:17.201909380 (14h9m11.632989511s) [upload=14777]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:35:17.201911984 (14h9m11.632992116s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:35:17.417157919 (14h9m11.84823805s) [upload=14777]: Done uploading segments... Aug 9 07:35:17.417175132 (14h9m11.848255263s) [upload=14777]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.865068189s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............X..X..X......X.......X.......................] duration: 6.380140001s FAIL (2025-08-09 07:37:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.380243188s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X....................X........................X........] duration: 5.563176675s FAIL (2025-08-09 07:37:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.563201112s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................X..........X........................X...] duration: 1.826764882s FAIL (2025-08-09 07:37:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 07:35:17.201671304 (14h9m11.632751425s) [upload=14777]: Starting upload Aug 9 07:35:17.201718444 (14h9m11.632798575s) [upload=14777]: Done starting upload Aug 9 07:35:17.201821532 (14h9m11.632901663s) [upload=14777]: Uploading segments... Aug 9 07:35:17.201864483 (14h9m11.632944614s) [upload=14777]: Got next segment. Inline: true Aug 9 07:35:17.201870205 (14h9m11.632950336s) [upload=14777]: Waiting for error group managing segments... Aug 9 07:35:17.201904821 (14h9m11.632984952s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:35:17.201909380 (14h9m11.632989511s) [upload=14777]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:35:17.201911984 (14h9m11.632992116s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:35:17.417157919 (14h9m11.84823805s) [upload=14777]: Done uploading segments... Aug 9 07:35:17.417175132 (14h9m11.848255263s) [upload=14777]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.826785731s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................X..........X........................X..] duration: 1.940528117s FAIL (2025-08-09 07:37:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 07:35:17.201671304 (14h9m11.632751425s) [upload=14777]: Starting upload Aug 9 07:35:17.201718444 (14h9m11.632798575s) [upload=14777]: Done starting upload Aug 9 07:35:17.201821532 (14h9m11.632901663s) [upload=14777]: Uploading segments... Aug 9 07:35:17.201864483 (14h9m11.632944614s) [upload=14777]: Got next segment. Inline: true Aug 9 07:35:17.201870205 (14h9m11.632950336s) [upload=14777]: Waiting for error group managing segments... Aug 9 07:35:17.201904821 (14h9m11.632984952s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:35:17.201909380 (14h9m11.632989511s) [upload=14777]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:35:17.201911984 (14h9m11.632992116s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:35:17.417157919 (14h9m11.84823805s) [upload=14777]: Done uploading segments... Aug 9 07:35:17.417175132 (14h9m11.848255263s) [upload=14777]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.940552963s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................X..X..X......X.......X......................] duration: 5.840417274s FAIL (2025-08-09 07:36:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.840444646s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X....................X........................X.......] duration: 5.820171259s FAIL (2025-08-09 07:36:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.82020774s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................X..X..X......X.......X.....................] duration: 6.229228527s FAIL (2025-08-09 07:36:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.229254467s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................X..........X........................X.] duration: 1.923093488s FAIL (2025-08-09 07:36:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 07:35:17.201671304 (14h9m11.632751425s) [upload=14777]: Starting upload Aug 9 07:35:17.201718444 (14h9m11.632798575s) [upload=14777]: Done starting upload Aug 9 07:35:17.201821532 (14h9m11.632901663s) [upload=14777]: Uploading segments... Aug 9 07:35:17.201864483 (14h9m11.632944614s) [upload=14777]: Got next segment. Inline: true Aug 9 07:35:17.201870205 (14h9m11.632950336s) [upload=14777]: Waiting for error group managing segments... Aug 9 07:35:17.201904821 (14h9m11.632984952s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:35:17.201909380 (14h9m11.632989511s) [upload=14777]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:35:17.201911984 (14h9m11.632992116s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:35:17.417157919 (14h9m11.84823805s) [upload=14777]: Done uploading segments... Aug 9 07:35:17.417175132 (14h9m11.848255263s) [upload=14777]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.923123287s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................X..X..X......X.......X....................] duration: 6.043233981s FAIL (2025-08-09 07:35:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.043261994s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X....................X........................X......] duration: 5.796069867s FAIL (2025-08-09 07:35:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.796098461s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................X..........X........................X] duration: 867.701343ms FAIL (2025-08-09 07:35:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 07:35:17.201671304 (14h9m11.632751425s) [upload=14777]: Starting upload Aug 9 07:35:17.201718444 (14h9m11.632798575s) [upload=14777]: Done starting upload Aug 9 07:35:17.201821532 (14h9m11.632901663s) [upload=14777]: Uploading segments... Aug 9 07:35:17.201864483 (14h9m11.632944614s) [upload=14777]: Got next segment. Inline: true Aug 9 07:35:17.201870205 (14h9m11.632950336s) [upload=14777]: Waiting for error group managing segments... Aug 9 07:35:17.201904821 (14h9m11.632984952s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:35:17.201909380 (14h9m11.632989511s) [upload=14777]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:35:17.201911984 (14h9m11.632992116s) [upload=14777]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:35:17.417157919 (14h9m11.84823805s) [upload=14777]: Done uploading segments... Aug 9 07:35:17.417175132 (14h9m11.848255263s) [upload=14777]: Upload finished. err: metaclient: Too Many Requests Duration:* 867.725248ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X..........X........................] duration: 1.873318655s FAIL (2025-08-09 07:35:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.873351998s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................X..X..X......X.......X...................] duration: 6.50348603s FAIL (2025-08-09 07:34:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.503599005s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X..........X.......................] duration: 1.849457289s FAIL (2025-08-09 07:34:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.849484452s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X....................X........................X.....] duration: 5.57316293s FAIL (2025-08-09 07:34:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.573198699s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................X..X..X......X.......X..................] duration: 6.407379906s FAIL (2025-08-09 07:34:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.407436774s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X..........X......................] duration: 1.829192509s FAIL (2025-08-09 07:33:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.829219832s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X....................X........................X....] duration: 5.823939831s FAIL (2025-08-09 07:33:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.82398069s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................X..X..X......X.......X.................] duration: 6.124097808s FAIL (2025-08-09 07:33:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.124136562s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X..........X.....................] duration: 1.803371282s FAIL (2025-08-09 07:32:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.803399456s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X..........X....................] duration: 1.835703632s FAIL (2025-08-09 07:32:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.83573336s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................X..X..X......X.......X................] duration: 5.949100613s FAIL (2025-08-09 07:32:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.949122395s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X....................X........................X...] duration: 5.710925196s FAIL (2025-08-09 07:32:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.710947929s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X..........X...................] duration: 1.858854678s FAIL (2025-08-09 07:32:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.858882642s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................X..........X..................] duration: 1.826078931s FAIL (2025-08-09 07:31:54 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.826131189s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X....................X........................X..] duration: 5.755005178s FAIL (2025-08-09 07:31:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.755036587s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................X..X..X......X.......X...............] duration: 6.395607846s FAIL (2025-08-09 07:31:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.395629869s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [........................X..X..X......X.......X..............] duration: 5.962573217s FAIL (2025-08-09 07:31:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.962590579s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X..........X.................] duration: 2.057016699s FAIL (2025-08-09 07:30:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 2.057045384s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X....................X........................X.] duration: 5.615573057s FAIL (2025-08-09 07:30:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.615599778s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X..........X................] duration: 1.892243504s FAIL (2025-08-09 07:30:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.892270305s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........................X..X..X......X.......X.............] duration: 6.424290977s FAIL (2025-08-09 07:30:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.424315514s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X....................X........................X] duration: 5.560185365s FAIL (2025-08-09 07:29:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 991b8db2ebc18cde52ec2f665e5232869b03cc2c98605fc99a7055e57c318971 != before 1e2d2432a81b2b2286ee734ea4920a624d92f9810761ad85e4233be65a221a33 ======LOGS====== Aug 9 07:29:54.072956519 (14h3m48.50403665s) [upload=14684]: Starting upload Aug 9 07:29:54.073004681 (14h3m48.504084812s) [upload=14684]: Done starting upload Aug 9 07:29:54.073135080 (14h3m48.504215201s) [upload=14684]: Uploading segments... Aug 9 07:29:54.073219702 (14h3m48.504299823s) [upload=14684]: Got next segment. Inline: false Aug 9 07:29:54.073263436 (14h3m48.504343567s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:29:54.073268215 (14h3m48.504348346s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:29:54.521851259 (14h3m48.95293139s) [seg_pos={0 0}, upload=14684]: Begin upload segment... Aug 9 07:29:54.521891206 (14h3m48.952971337s) [seg_pos={0 0}, upload=14684]: Done begin upload segment. Aug 9 07:29:54.521916254 (14h3m48.952996385s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522013891 (14h3m48.953094022s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522315618 (14h3m48.953395749s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522602337 (14h3m48.953682468s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522657733 (14h3m48.953737864s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.522883324 (14h3m48.953963445s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523076504 (14h3m48.954156635s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523174211 (14h3m48.954254342s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523408599 (14h3m48.95448872s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.523470818 (14h3m48.954550949s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524089172 (14h3m48.955169303s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524298672 (14h3m48.955378803s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524444782 (14h3m48.955524913s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.524965288 (14h3m48.956045419s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525234312 (14h3m48.956314443s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.525973466 (14h3m48.957053587s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526222483 (14h3m48.957302614s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:54.526598963 (14h3m48.957679084s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Uploading piece... Aug 9 07:29:56.041724208 (14h3m50.472804339s) [upload=14684]: Next returned nil segment Aug 9 07:29:56.041740099 (14h3m50.47282023s) [upload=14684]: Waiting for error group managing segments... Aug 9 07:29:56.445259603 (14h3m50.876339734s) [noise=true , address=77.90.31.14:4675 , piece_id=4CKVLIE73AR727LNTPXPYOC63OJRGRA2PQTDMY43UPLZXZILMZ7Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.450238929 (14h3m50.881319091s) [noise=true , address=45.91.116.4:4969 , piece_id=6G2WGANYVOEAQI5H4TI26YXBKYIXPR6YPL4Y6B5FBQA2ZRODHFBQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.465455043 (14h3m50.896535164s) [noise=true , address=45.91.116.9:4276 , piece_id=LXCMHFME2JORK3P55AMNLU5FWO6FOHCBCHO6NMVP3F73POEWAFEA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469187675 (14h3m50.900267806s) [noise=true , address=104.36.82.23:4450 , piece_id=WT5GFG35VAUCEMTSVVJOYML4XKCS6R7FA3LG6EGFRN2HBFAIDR4Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.469783464 (14h3m50.900863595s) [noise=true , address=104.36.82.6:4898 , piece_id=53GM7CXLMAGUUOJ3O6N7MX56A4SGAOGKBLZV2Z7GYYVARF4HSD6A, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.470306505 (14h3m50.901386636s) [noise=true , address=104.36.82.37:4671 , piece_id=5KUSOMLN2EXGIN52SHER6IT2HPXZEKKX4N5JFECYHPMSVGY6WBLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.471439473 (14h3m50.902519604s) [noise=true , address=104.36.82.36:4639 , piece_id=DMZ4FYWZXXG7C5T62YGOBHHA6D7AAAEYC7E6FDR4W6N27CSTFR2Q, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.476826960 (14h3m50.907907081s) [noise=true , address=104.36.82.60:4352 , piece_id=2S2OB4QMUZSSIVNPQ4QJVAV2HUEMYGT2RC27B6YHTZZUEKU334XQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.482521336 (14h3m50.913601457s) [noise=true , address=104.36.82.35:4605 , piece_id=PEXL6UNS3LYHR2TKAWY4JF4FIJ3CEMTRV4C7GWONK32JEYAYCDLQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484473371 (14h3m50.915553502s) [noise=true , address=104.36.82.45:4931 , piece_id=S7PCWHBDQNW46FC6RHQTJNJLSUQPUJJUQYOYI33XHCTLTGVE2GUA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.484520542 (14h3m50.915600703s) [noise=true , address=104.36.82.47:4548 , piece_id=FFUK6IVNGNOSURQQ63UGIMVUMCDV2SXPKXU7FFOT4DATGESOSMHA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.500269895 (14h3m50.931350026s) [noise=true , address=104.36.82.12:4172 , piece_id=R2UIYG5PBIMKC6IQ7E6Y6SPA7SEI6PO2JQLFWAJOKSUQCWAFX2OA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.506487591 (14h3m50.937567722s) [noise=true , address=104.36.82.13:4192 , piece_id=CYKWCGRXRQRAFKGGSKXPOHOXZFRRY6JLYZ5NZPL7PL6W6UITU5KA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510595772 (14h3m50.941675903s) [noise=true , address=104.36.82.59:4319 , piece_id=VGK4DOAIFJH4A4YMCDRPDS2XCTRCK6O2YGTHELQQ575RW3NLIGVA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: <nil> Aug 9 07:29:56.510634255 (14h3m50.941714386s) [seg_pos={0 0}, upload=14684]: Segment reached optimal threshold of 14 pieces. Aug 9 07:29:56.510806765 (14h3m50.941886896s) [noise=true , address=104.36.82.16:4317 , piece_id=4OE2XWTFXRDK6W7GHGVJELCJAS6OVJC6EPZLZUQKZ7GJW44SELYQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FdJoeW4vejiQx1QkQ1nCkApgQaW6fB66d2usAYVfFuK1PPmRP): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510902659 (14h3m50.94198279s) [noise=true , address=104.36.82.61:4384 , piece_id=6GXTLX3CYFGWLM6DQJLWUPAUYNH4UKKATV3MJ5EJTXDOP6OTK2NA, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FygMGuy8K7NzctYWRxRNmxZqrXNFfJLkNyHWLb4dhwumJKDkr): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510926815 (14h3m50.942006947s) [noise=true , address=104.36.82.17:4346 , piece_id=A5T3FQ6Q7PZ6N2KIHUDJVAF757OGUHD74VKZEXMEZXJO6WWPF2MQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kuxXNRsBsP6izniQ2AcT4DJRW2RYWdJurSPYh5h8TF32pwzza): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.510990297 (14h3m50.942070428s) [noise=true , address=104.36.82.20:4456 , piece_id=AGFM4FOPT7Q23B7BB5R4PKWCSI6PI4SZDAMZTFVA52P7UYZJHTKQ, seg_pos={0 0}, upload=14684]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TmBh7KCuUydq7ZnVtJR4cWKa6ktWH4me19r8vGdggGxohDUyv): protocol: expected piece hash; context canceled; EOF Aug 9 07:29:56.511011157 (14h3m50.942091288s) [seg_pos={0 0}, upload=14684]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:29:56.511097191 (14h3m50.942177322s) [upload=14684]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:29:56.511103203 (14h3m50.942183334s) [upload=14684]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:29:56.807252481 (14h3m51.238332602s) [upload=14684]: Done uploading segments... Aug 9 07:29:56.807269464 (14h3m51.238349595s) [upload=14684]: Upload finished. err: <nil> Duration: 5.56021473s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X..........X...............] duration: 1.931516799s FAIL (2025-08-09 07:29:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.931544332s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X....................X........................] duration: 5.800148455s FAIL (2025-08-09 07:29:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e3ff044d6ae4076192b1e1fe513b8dabe6c767a5c3203dea27eaa2ec31688bdd != before 663c2bc3be3b2c93bc26181187625e6a4f7893b1381782a3fe391ac4fca8ac68 ======LOGS====== Aug 9 07:12:10.752933843 (13h46m5.184013974s) [upload=14368]: Starting upload Aug 9 07:12:10.752992164 (13h46m5.184072285s) [upload=14368]: Done starting upload Aug 9 07:12:10.753168141 (13h46m5.184248273s) [upload=14368]: Uploading segments... Aug 9 07:12:10.753297318 (13h46m5.184377449s) [upload=14368]: Got next segment. Inline: false Aug 9 07:12:10.753326153 (13h46m5.184406274s) [upload=14368]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:12:10.753330122 (13h46m5.184410253s) [upload=14368]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:12:11.207655625 (13h46m5.638735756s) [seg_pos={0 0}, upload=14368]: Begin upload segment... Aug 9 07:12:11.207720700 (13h46m5.638800831s) [seg_pos={0 0}, upload=14368]: Done begin upload segment. Aug 9 07:12:11.207743194 (13h46m5.638823325s) [noise=true , address=104.36.82.44:4896 , piece_id=OHFYRMATCJBA2K3HRJ77ZJTOJ3HT7MK7N5JC7NZ52PXEJ35JKWHQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.207765646 (13h46m5.638845767s) [noise=true , address=64.49.13.27:4961 , piece_id=GYEU7TP4OXYEYFXENUVQRUA4B6Q75NU52NLSADC2AQ3477QHTBSA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208227149 (13h46m5.63930728s) [noise=true , address=64.49.13.139:4479 , piece_id=5IUHFINOLUATEGGJW3XJL4KPTUV2NC4YWICUUKHFEKBHSMRIA2RA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208364321 (13h46m5.639444463s) [noise=true , address=104.36.82.60:4370 , piece_id=T3P6UIMSI73BKLVUYXHWEKOXTEXWVOGFSG2NSVJ3GK6M2LBXSKZA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208698120 (13h46m5.639778251s) [noise=true , address=104.36.82.8:4981 , piece_id=FOIG62XE46FFMD5UHH2SN6SX7KGFEV5YK72DM6WBFROHGC665ZPA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209043852 (13h46m5.640123983s) [noise=true , address=104.36.82.59:4324 , piece_id=ZWHTHIGUKE4BGL7XQGQ7SSQOKV2AS7KCKRU3IG7XOPN75FJCCIRA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209320301 (13h46m5.640400432s) [noise=true , address=64.49.13.169:4715 , piece_id=F4XFTIMZVJQFZ7WPSM65CRORDOAJ5RXNFD2LW7JWRUISIB2OQQTQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209683986 (13h46m5.640764117s) [noise=true , address=64.49.13.170:4724 , piece_id=NGADM4QFZPZV2ZQ3T7D55CW5YASOZA6NCPBA2RECAJQ543YBBXHA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210169786 (13h46m5.641249917s) [noise=true , address=104.36.82.60:4371 , piece_id=7A4ZKM7IMWTIEPXVP4VWYWXEZZ322WJPV7FNFDHS67OFVQK4QASA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210381952 (13h46m5.641462083s) [noise=true , address=104.36.82.61:4404 , piece_id=42LT6BXLMKSGA6Q7P3IR4SZZZLAMKUHJO4QWZYULTV5AFMCHPL3Q, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210636339 (13h46m5.64171647s) [noise=true , address=104.36.82.43:4847 , piece_id=5KKBDZL65I2KSGRPO3OHVV4UBY7XVV2NLTTKOY2BP65ESCKFSSRA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210856220 (13h46m5.641936351s) [noise=true , address=104.36.82.16:4316 , piece_id=4NPL3A5HFVERECAHTLPRIGBWPFA7QW4L2WTX55W7L2DCYUEG75XA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211079006 (13h46m5.642159127s) [noise=true , address=104.36.82.14:4237 , piece_id=RLIE5EYM6A7VNRDTTFXJTUV7WJKJ5QDKZEAOMUZYA3QFNWXPQKNQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211304737 (13h46m5.642384868s) [noise=true , address=45.91.116.7:4031 , piece_id=I3M2I7TE2HH6KKY4R7VB5PSNZJXAVCK36UULZ2BLWNKWKTYTJIQQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211491064 (13h46m5.642571195s) [noise=true , address=104.36.82.37:4669 , piece_id=KK2GNVWD6EUIBG77ZIEAH3ZBDDXO4PX2DFCFPWDTCT4KFTG5ESLA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211981352 (13h46m5.643061483s) [noise=true , address=104.36.82.15:4292 , piece_id=3FMM25ACGF4RPBO66JM4PMB2BC2DCVGCR3JCMWCONRBYSODONGFA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.212224658 (13h46m5.643304789s) [noise=true , address=104.36.82.3:4795 , piece_id=DDRQ7JUPFC5FTKGOT6J4YHIXJXEGPOVQYPV3EQ35RDGHVU3H4OUQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.212477902 (13h46m5.643558033s) [noise=true , address=104.36.82.39:4751 , piece_id=ZM64XI6VX7YENY6WSGRMS34XAATUXGEVWPISJMYOXXQB5LLSBJYA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:12.328114668 (13h46m6.759194799s) [upload=14368]: Next returned nil segment Aug 9 07:12:12.328130238 (13h46m6.759210359s) [upload=14368]: Waiting for error group managing segments... Aug 9 07:12:12.615123755 (13h46m7.046203886s) [noise=true , address=64.49.13.170:4724 , piece_id=NGADM4QFZPZV2ZQ3T7D55CW5YASOZA6NCPBA2RECAJQ543YBBXHA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.633849403 (13h46m7.064929534s) [noise=true , address=64.49.13.139:4479 , piece_id=5IUHFINOLUATEGGJW3XJL4KPTUV2NC4YWICUUKHFEKBHSMRIA2RA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.637449872 (13h46m7.068530003s) [noise=true , address=64.49.13.27:4961 , piece_id=GYEU7TP4OXYEYFXENUVQRUA4B6Q75NU52NLSADC2AQ3477QHTBSA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.650403255 (13h46m7.081483387s) [noise=true , address=64.49.13.169:4715 , piece_id=F4XFTIMZVJQFZ7WPSM65CRORDOAJ5RXNFD2LW7JWRUISIB2OQQTQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.137896401 (13h46m7.568976532s) [noise=true , address=104.36.82.60:4371 , piece_id=7A4ZKM7IMWTIEPXVP4VWYWXEZZ322WJPV7FNFDHS67OFVQK4QASA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.139324974 (13h46m7.570405095s) [noise=true , address=104.36.82.8:4981 , piece_id=FOIG62XE46FFMD5UHH2SN6SX7KGFEV5YK72DM6WBFROHGC665ZPA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.146401663 (13h46m7.577481794s) [noise=true , address=45.91.116.7:4031 , piece_id=I3M2I7TE2HH6KKY4R7VB5PSNZJXAVCK36UULZ2BLWNKWKTYTJIQQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.161160482 (13h46m7.592240603s) [noise=true , address=104.36.82.43:4847 , piece_id=5KKBDZL65I2KSGRPO3OHVV4UBY7XVV2NLTTKOY2BP65ESCKFSSRA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.165737920 (13h46m7.596818041s) [noise=true , address=104.36.82.15:4292 , piece_id=3FMM25ACGF4RPBO66JM4PMB2BC2DCVGCR3JCMWCONRBYSODONGFA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.168998608 (13h46m7.600078729s) [noise=true , address=104.36.82.60:4370 , piece_id=T3P6UIMSI73BKLVUYXHWEKOXTEXWVOGFSG2NSVJ3GK6M2LBXSKZA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.209005594 (13h46m7.640085725s) [noise=true , address=104.36.82.59:4324 , piece_id=ZWHTHIGUKE4BGL7XQGQ7SSQOKV2AS7KCKRU3IG7XOPN75FJCCIRA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.229451654 (13h46m7.660531785s) [noise=true , address=104.36.82.3:4795 , piece_id=DDRQ7JUPFC5FTKGOT6J4YHIXJXEGPOVQYPV3EQ35RDGHVU3H4OUQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.238304101 (13h46m7.669384232s) [noise=true , address=104.36.82.61:4404 , piece_id=42LT6BXLMKSGA6Q7P3IR4SZZZLAMKUHJO4QWZYULTV5AFMCHPL3Q, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.245475733 (13h46m7.676555854s) [noise=true , address=104.36.82.37:4669 , piece_id=KK2GNVWD6EUIBG77ZIEAH3ZBDDXO4PX2DFCFPWDTCT4KFTG5ESLA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.245503065 (13h46m7.676583196s) [seg_pos={0 0}, upload=14368]: Segment reached optimal threshold of 14 pieces. Aug 9 07:12:13.245606703 (13h46m7.676686834s) [noise=true , address=104.36.82.14:4237 , piece_id=RLIE5EYM6A7VNRDTTFXJTUV7WJKJ5QDKZEAOMUZYA3QFNWXPQKNQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245678009 (13h46m7.67675814s) [noise=true , address=104.36.82.16:4316 , piece_id=4NPL3A5HFVERECAHTLPRIGBWPFA7QW4L2WTX55W7L2DCYUEG75XA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PPnXDzwXDVDKemw483GXuzQrPN48JJrjeeD7az2NPNawHNaXf): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245759245 (13h46m7.676839366s) [noise=true , address=104.36.82.44:4896 , piece_id=OHFYRMATCJBA2K3HRJ77ZJTOJ3HT7MK7N5JC7NZ52PXEJ35JKWHQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ndSgjgfZMVQni5tw1ugdTrhJt4c4NagdvgYTbSbjatYYtfu1o): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245792349 (13h46m7.67687247s) [noise=true , address=104.36.82.39:4751 , piece_id=ZM64XI6VX7YENY6WSGRMS34XAATUXGEVWPISJMYOXXQB5LLSBJYA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128StmvpG1KYRXeu3gzEEfRmYZgR5ACJRf2xGZcEiZHsfk1KNyr): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245815813 (13h46m7.676895944s) [seg_pos={0 0}, upload=14368]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:12:13.245901858 (13h46m7.676981979s) [upload=14368]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:12:13.245907359 (13h46m7.67698749s) [upload=14368]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:12:13.789706183 (13h46m8.220786304s) [upload=14368]: Done uploading segments... Aug 9 07:12:13.789724909 (13h46m8.22080503s) [upload=14368]: Upload finished. err: <nil> Duration: 5.800191147s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........................X..X..X......X.......X............] duration: 5.857277776s FAIL (2025-08-09 07:29:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.857297043s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X..........X..............] duration: 2.51324541s FAIL (2025-08-09 07:29:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 2.513287811s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X....................X.......................] duration: 5.786686091s FAIL (2025-08-09 07:28:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e3ff044d6ae4076192b1e1fe513b8dabe6c767a5c3203dea27eaa2ec31688bdd != before 663c2bc3be3b2c93bc26181187625e6a4f7893b1381782a3fe391ac4fca8ac68 ======LOGS====== Aug 9 07:12:10.752933843 (13h46m5.184013974s) [upload=14368]: Starting upload Aug 9 07:12:10.752992164 (13h46m5.184072285s) [upload=14368]: Done starting upload Aug 9 07:12:10.753168141 (13h46m5.184248273s) [upload=14368]: Uploading segments... Aug 9 07:12:10.753297318 (13h46m5.184377449s) [upload=14368]: Got next segment. Inline: false Aug 9 07:12:10.753326153 (13h46m5.184406274s) [upload=14368]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:12:10.753330122 (13h46m5.184410253s) [upload=14368]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:12:11.207655625 (13h46m5.638735756s) [seg_pos={0 0}, upload=14368]: Begin upload segment... Aug 9 07:12:11.207720700 (13h46m5.638800831s) [seg_pos={0 0}, upload=14368]: Done begin upload segment. Aug 9 07:12:11.207743194 (13h46m5.638823325s) [noise=true , address=104.36.82.44:4896 , piece_id=OHFYRMATCJBA2K3HRJ77ZJTOJ3HT7MK7N5JC7NZ52PXEJ35JKWHQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.207765646 (13h46m5.638845767s) [noise=true , address=64.49.13.27:4961 , piece_id=GYEU7TP4OXYEYFXENUVQRUA4B6Q75NU52NLSADC2AQ3477QHTBSA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208227149 (13h46m5.63930728s) [noise=true , address=64.49.13.139:4479 , piece_id=5IUHFINOLUATEGGJW3XJL4KPTUV2NC4YWICUUKHFEKBHSMRIA2RA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208364321 (13h46m5.639444463s) [noise=true , address=104.36.82.60:4370 , piece_id=T3P6UIMSI73BKLVUYXHWEKOXTEXWVOGFSG2NSVJ3GK6M2LBXSKZA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208698120 (13h46m5.639778251s) [noise=true , address=104.36.82.8:4981 , piece_id=FOIG62XE46FFMD5UHH2SN6SX7KGFEV5YK72DM6WBFROHGC665ZPA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209043852 (13h46m5.640123983s) [noise=true , address=104.36.82.59:4324 , piece_id=ZWHTHIGUKE4BGL7XQGQ7SSQOKV2AS7KCKRU3IG7XOPN75FJCCIRA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209320301 (13h46m5.640400432s) [noise=true , address=64.49.13.169:4715 , piece_id=F4XFTIMZVJQFZ7WPSM65CRORDOAJ5RXNFD2LW7JWRUISIB2OQQTQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209683986 (13h46m5.640764117s) [noise=true , address=64.49.13.170:4724 , piece_id=NGADM4QFZPZV2ZQ3T7D55CW5YASOZA6NCPBA2RECAJQ543YBBXHA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210169786 (13h46m5.641249917s) [noise=true , address=104.36.82.60:4371 , piece_id=7A4ZKM7IMWTIEPXVP4VWYWXEZZ322WJPV7FNFDHS67OFVQK4QASA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210381952 (13h46m5.641462083s) [noise=true , address=104.36.82.61:4404 , piece_id=42LT6BXLMKSGA6Q7P3IR4SZZZLAMKUHJO4QWZYULTV5AFMCHPL3Q, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210636339 (13h46m5.64171647s) [noise=true , address=104.36.82.43:4847 , piece_id=5KKBDZL65I2KSGRPO3OHVV4UBY7XVV2NLTTKOY2BP65ESCKFSSRA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210856220 (13h46m5.641936351s) [noise=true , address=104.36.82.16:4316 , piece_id=4NPL3A5HFVERECAHTLPRIGBWPFA7QW4L2WTX55W7L2DCYUEG75XA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211079006 (13h46m5.642159127s) [noise=true , address=104.36.82.14:4237 , piece_id=RLIE5EYM6A7VNRDTTFXJTUV7WJKJ5QDKZEAOMUZYA3QFNWXPQKNQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211304737 (13h46m5.642384868s) [noise=true , address=45.91.116.7:4031 , piece_id=I3M2I7TE2HH6KKY4R7VB5PSNZJXAVCK36UULZ2BLWNKWKTYTJIQQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211491064 (13h46m5.642571195s) [noise=true , address=104.36.82.37:4669 , piece_id=KK2GNVWD6EUIBG77ZIEAH3ZBDDXO4PX2DFCFPWDTCT4KFTG5ESLA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211981352 (13h46m5.643061483s) [noise=true , address=104.36.82.15:4292 , piece_id=3FMM25ACGF4RPBO66JM4PMB2BC2DCVGCR3JCMWCONRBYSODONGFA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.212224658 (13h46m5.643304789s) [noise=true , address=104.36.82.3:4795 , piece_id=DDRQ7JUPFC5FTKGOT6J4YHIXJXEGPOVQYPV3EQ35RDGHVU3H4OUQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.212477902 (13h46m5.643558033s) [noise=true , address=104.36.82.39:4751 , piece_id=ZM64XI6VX7YENY6WSGRMS34XAATUXGEVWPISJMYOXXQB5LLSBJYA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:12.328114668 (13h46m6.759194799s) [upload=14368]: Next returned nil segment Aug 9 07:12:12.328130238 (13h46m6.759210359s) [upload=14368]: Waiting for error group managing segments... Aug 9 07:12:12.615123755 (13h46m7.046203886s) [noise=true , address=64.49.13.170:4724 , piece_id=NGADM4QFZPZV2ZQ3T7D55CW5YASOZA6NCPBA2RECAJQ543YBBXHA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.633849403 (13h46m7.064929534s) [noise=true , address=64.49.13.139:4479 , piece_id=5IUHFINOLUATEGGJW3XJL4KPTUV2NC4YWICUUKHFEKBHSMRIA2RA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.637449872 (13h46m7.068530003s) [noise=true , address=64.49.13.27:4961 , piece_id=GYEU7TP4OXYEYFXENUVQRUA4B6Q75NU52NLSADC2AQ3477QHTBSA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.650403255 (13h46m7.081483387s) [noise=true , address=64.49.13.169:4715 , piece_id=F4XFTIMZVJQFZ7WPSM65CRORDOAJ5RXNFD2LW7JWRUISIB2OQQTQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.137896401 (13h46m7.568976532s) [noise=true , address=104.36.82.60:4371 , piece_id=7A4ZKM7IMWTIEPXVP4VWYWXEZZ322WJPV7FNFDHS67OFVQK4QASA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.139324974 (13h46m7.570405095s) [noise=true , address=104.36.82.8:4981 , piece_id=FOIG62XE46FFMD5UHH2SN6SX7KGFEV5YK72DM6WBFROHGC665ZPA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.146401663 (13h46m7.577481794s) [noise=true , address=45.91.116.7:4031 , piece_id=I3M2I7TE2HH6KKY4R7VB5PSNZJXAVCK36UULZ2BLWNKWKTYTJIQQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.161160482 (13h46m7.592240603s) [noise=true , address=104.36.82.43:4847 , piece_id=5KKBDZL65I2KSGRPO3OHVV4UBY7XVV2NLTTKOY2BP65ESCKFSSRA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.165737920 (13h46m7.596818041s) [noise=true , address=104.36.82.15:4292 , piece_id=3FMM25ACGF4RPBO66JM4PMB2BC2DCVGCR3JCMWCONRBYSODONGFA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.168998608 (13h46m7.600078729s) [noise=true , address=104.36.82.60:4370 , piece_id=T3P6UIMSI73BKLVUYXHWEKOXTEXWVOGFSG2NSVJ3GK6M2LBXSKZA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.209005594 (13h46m7.640085725s) [noise=true , address=104.36.82.59:4324 , piece_id=ZWHTHIGUKE4BGL7XQGQ7SSQOKV2AS7KCKRU3IG7XOPN75FJCCIRA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.229451654 (13h46m7.660531785s) [noise=true , address=104.36.82.3:4795 , piece_id=DDRQ7JUPFC5FTKGOT6J4YHIXJXEGPOVQYPV3EQ35RDGHVU3H4OUQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.238304101 (13h46m7.669384232s) [noise=true , address=104.36.82.61:4404 , piece_id=42LT6BXLMKSGA6Q7P3IR4SZZZLAMKUHJO4QWZYULTV5AFMCHPL3Q, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.245475733 (13h46m7.676555854s) [noise=true , address=104.36.82.37:4669 , piece_id=KK2GNVWD6EUIBG77ZIEAH3ZBDDXO4PX2DFCFPWDTCT4KFTG5ESLA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.245503065 (13h46m7.676583196s) [seg_pos={0 0}, upload=14368]: Segment reached optimal threshold of 14 pieces. Aug 9 07:12:13.245606703 (13h46m7.676686834s) [noise=true , address=104.36.82.14:4237 , piece_id=RLIE5EYM6A7VNRDTTFXJTUV7WJKJ5QDKZEAOMUZYA3QFNWXPQKNQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245678009 (13h46m7.67675814s) [noise=true , address=104.36.82.16:4316 , piece_id=4NPL3A5HFVERECAHTLPRIGBWPFA7QW4L2WTX55W7L2DCYUEG75XA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PPnXDzwXDVDKemw483GXuzQrPN48JJrjeeD7az2NPNawHNaXf): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245759245 (13h46m7.676839366s) [noise=true , address=104.36.82.44:4896 , piece_id=OHFYRMATCJBA2K3HRJ77ZJTOJ3HT7MK7N5JC7NZ52PXEJ35JKWHQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ndSgjgfZMVQni5tw1ugdTrhJt4c4NagdvgYTbSbjatYYtfu1o): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245792349 (13h46m7.67687247s) [noise=true , address=104.36.82.39:4751 , piece_id=ZM64XI6VX7YENY6WSGRMS34XAATUXGEVWPISJMYOXXQB5LLSBJYA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128StmvpG1KYRXeu3gzEEfRmYZgR5ACJRf2xGZcEiZHsfk1KNyr): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245815813 (13h46m7.676895944s) [seg_pos={0 0}, upload=14368]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:12:13.245901858 (13h46m7.676981979s) [upload=14368]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:12:13.245907359 (13h46m7.67698749s) [upload=14368]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:12:13.789706183 (13h46m8.220786304s) [upload=14368]: Done uploading segments... Aug 9 07:12:13.789724909 (13h46m8.22080503s) [upload=14368]: Upload finished. err: <nil> Duration: 5.786709806s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........................X..X..X......X.......X...........] duration: 6.269812338s FAIL (2025-08-09 07:28:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.269827328s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X..........X.............] duration: 1.838872157s FAIL (2025-08-09 07:28:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.838930578s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X....................X......................] duration: 5.521386823s FAIL (2025-08-09 07:28:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e3ff044d6ae4076192b1e1fe513b8dabe6c767a5c3203dea27eaa2ec31688bdd != before 663c2bc3be3b2c93bc26181187625e6a4f7893b1381782a3fe391ac4fca8ac68 ======LOGS====== Aug 9 07:12:10.752933843 (13h46m5.184013974s) [upload=14368]: Starting upload Aug 9 07:12:10.752992164 (13h46m5.184072285s) [upload=14368]: Done starting upload Aug 9 07:12:10.753168141 (13h46m5.184248273s) [upload=14368]: Uploading segments... Aug 9 07:12:10.753297318 (13h46m5.184377449s) [upload=14368]: Got next segment. Inline: false Aug 9 07:12:10.753326153 (13h46m5.184406274s) [upload=14368]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:12:10.753330122 (13h46m5.184410253s) [upload=14368]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:12:11.207655625 (13h46m5.638735756s) [seg_pos={0 0}, upload=14368]: Begin upload segment... Aug 9 07:12:11.207720700 (13h46m5.638800831s) [seg_pos={0 0}, upload=14368]: Done begin upload segment. Aug 9 07:12:11.207743194 (13h46m5.638823325s) [noise=true , address=104.36.82.44:4896 , piece_id=OHFYRMATCJBA2K3HRJ77ZJTOJ3HT7MK7N5JC7NZ52PXEJ35JKWHQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.207765646 (13h46m5.638845767s) [noise=true , address=64.49.13.27:4961 , piece_id=GYEU7TP4OXYEYFXENUVQRUA4B6Q75NU52NLSADC2AQ3477QHTBSA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208227149 (13h46m5.63930728s) [noise=true , address=64.49.13.139:4479 , piece_id=5IUHFINOLUATEGGJW3XJL4KPTUV2NC4YWICUUKHFEKBHSMRIA2RA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208364321 (13h46m5.639444463s) [noise=true , address=104.36.82.60:4370 , piece_id=T3P6UIMSI73BKLVUYXHWEKOXTEXWVOGFSG2NSVJ3GK6M2LBXSKZA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208698120 (13h46m5.639778251s) [noise=true , address=104.36.82.8:4981 , piece_id=FOIG62XE46FFMD5UHH2SN6SX7KGFEV5YK72DM6WBFROHGC665ZPA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209043852 (13h46m5.640123983s) [noise=true , address=104.36.82.59:4324 , piece_id=ZWHTHIGUKE4BGL7XQGQ7SSQOKV2AS7KCKRU3IG7XOPN75FJCCIRA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209320301 (13h46m5.640400432s) [noise=true , address=64.49.13.169:4715 , piece_id=F4XFTIMZVJQFZ7WPSM65CRORDOAJ5RXNFD2LW7JWRUISIB2OQQTQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209683986 (13h46m5.640764117s) [noise=true , address=64.49.13.170:4724 , piece_id=NGADM4QFZPZV2ZQ3T7D55CW5YASOZA6NCPBA2RECAJQ543YBBXHA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210169786 (13h46m5.641249917s) [noise=true , address=104.36.82.60:4371 , piece_id=7A4ZKM7IMWTIEPXVP4VWYWXEZZ322WJPV7FNFDHS67OFVQK4QASA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210381952 (13h46m5.641462083s) [noise=true , address=104.36.82.61:4404 , piece_id=42LT6BXLMKSGA6Q7P3IR4SZZZLAMKUHJO4QWZYULTV5AFMCHPL3Q, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210636339 (13h46m5.64171647s) [noise=true , address=104.36.82.43:4847 , piece_id=5KKBDZL65I2KSGRPO3OHVV4UBY7XVV2NLTTKOY2BP65ESCKFSSRA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210856220 (13h46m5.641936351s) [noise=true , address=104.36.82.16:4316 , piece_id=4NPL3A5HFVERECAHTLPRIGBWPFA7QW4L2WTX55W7L2DCYUEG75XA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211079006 (13h46m5.642159127s) [noise=true , address=104.36.82.14:4237 , piece_id=RLIE5EYM6A7VNRDTTFXJTUV7WJKJ5QDKZEAOMUZYA3QFNWXPQKNQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211304737 (13h46m5.642384868s) [noise=true , address=45.91.116.7:4031 , piece_id=I3M2I7TE2HH6KKY4R7VB5PSNZJXAVCK36UULZ2BLWNKWKTYTJIQQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211491064 (13h46m5.642571195s) [noise=true , address=104.36.82.37:4669 , piece_id=KK2GNVWD6EUIBG77ZIEAH3ZBDDXO4PX2DFCFPWDTCT4KFTG5ESLA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211981352 (13h46m5.643061483s) [noise=true , address=104.36.82.15:4292 , piece_id=3FMM25ACGF4RPBO66JM4PMB2BC2DCVGCR3JCMWCONRBYSODONGFA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.212224658 (13h46m5.643304789s) [noise=true , address=104.36.82.3:4795 , piece_id=DDRQ7JUPFC5FTKGOT6J4YHIXJXEGPOVQYPV3EQ35RDGHVU3H4OUQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.212477902 (13h46m5.643558033s) [noise=true , address=104.36.82.39:4751 , piece_id=ZM64XI6VX7YENY6WSGRMS34XAATUXGEVWPISJMYOXXQB5LLSBJYA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:12.328114668 (13h46m6.759194799s) [upload=14368]: Next returned nil segment Aug 9 07:12:12.328130238 (13h46m6.759210359s) [upload=14368]: Waiting for error group managing segments... Aug 9 07:12:12.615123755 (13h46m7.046203886s) [noise=true , address=64.49.13.170:4724 , piece_id=NGADM4QFZPZV2ZQ3T7D55CW5YASOZA6NCPBA2RECAJQ543YBBXHA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.633849403 (13h46m7.064929534s) [noise=true , address=64.49.13.139:4479 , piece_id=5IUHFINOLUATEGGJW3XJL4KPTUV2NC4YWICUUKHFEKBHSMRIA2RA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.637449872 (13h46m7.068530003s) [noise=true , address=64.49.13.27:4961 , piece_id=GYEU7TP4OXYEYFXENUVQRUA4B6Q75NU52NLSADC2AQ3477QHTBSA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.650403255 (13h46m7.081483387s) [noise=true , address=64.49.13.169:4715 , piece_id=F4XFTIMZVJQFZ7WPSM65CRORDOAJ5RXNFD2LW7JWRUISIB2OQQTQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.137896401 (13h46m7.568976532s) [noise=true , address=104.36.82.60:4371 , piece_id=7A4ZKM7IMWTIEPXVP4VWYWXEZZ322WJPV7FNFDHS67OFVQK4QASA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.139324974 (13h46m7.570405095s) [noise=true , address=104.36.82.8:4981 , piece_id=FOIG62XE46FFMD5UHH2SN6SX7KGFEV5YK72DM6WBFROHGC665ZPA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.146401663 (13h46m7.577481794s) [noise=true , address=45.91.116.7:4031 , piece_id=I3M2I7TE2HH6KKY4R7VB5PSNZJXAVCK36UULZ2BLWNKWKTYTJIQQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.161160482 (13h46m7.592240603s) [noise=true , address=104.36.82.43:4847 , piece_id=5KKBDZL65I2KSGRPO3OHVV4UBY7XVV2NLTTKOY2BP65ESCKFSSRA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.165737920 (13h46m7.596818041s) [noise=true , address=104.36.82.15:4292 , piece_id=3FMM25ACGF4RPBO66JM4PMB2BC2DCVGCR3JCMWCONRBYSODONGFA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.168998608 (13h46m7.600078729s) [noise=true , address=104.36.82.60:4370 , piece_id=T3P6UIMSI73BKLVUYXHWEKOXTEXWVOGFSG2NSVJ3GK6M2LBXSKZA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.209005594 (13h46m7.640085725s) [noise=true , address=104.36.82.59:4324 , piece_id=ZWHTHIGUKE4BGL7XQGQ7SSQOKV2AS7KCKRU3IG7XOPN75FJCCIRA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.229451654 (13h46m7.660531785s) [noise=true , address=104.36.82.3:4795 , piece_id=DDRQ7JUPFC5FTKGOT6J4YHIXJXEGPOVQYPV3EQ35RDGHVU3H4OUQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.238304101 (13h46m7.669384232s) [noise=true , address=104.36.82.61:4404 , piece_id=42LT6BXLMKSGA6Q7P3IR4SZZZLAMKUHJO4QWZYULTV5AFMCHPL3Q, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.245475733 (13h46m7.676555854s) [noise=true , address=104.36.82.37:4669 , piece_id=KK2GNVWD6EUIBG77ZIEAH3ZBDDXO4PX2DFCFPWDTCT4KFTG5ESLA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.245503065 (13h46m7.676583196s) [seg_pos={0 0}, upload=14368]: Segment reached optimal threshold of 14 pieces. Aug 9 07:12:13.245606703 (13h46m7.676686834s) [noise=true , address=104.36.82.14:4237 , piece_id=RLIE5EYM6A7VNRDTTFXJTUV7WJKJ5QDKZEAOMUZYA3QFNWXPQKNQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245678009 (13h46m7.67675814s) [noise=true , address=104.36.82.16:4316 , piece_id=4NPL3A5HFVERECAHTLPRIGBWPFA7QW4L2WTX55W7L2DCYUEG75XA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PPnXDzwXDVDKemw483GXuzQrPN48JJrjeeD7az2NPNawHNaXf): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245759245 (13h46m7.676839366s) [noise=true , address=104.36.82.44:4896 , piece_id=OHFYRMATCJBA2K3HRJ77ZJTOJ3HT7MK7N5JC7NZ52PXEJ35JKWHQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ndSgjgfZMVQni5tw1ugdTrhJt4c4NagdvgYTbSbjatYYtfu1o): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245792349 (13h46m7.67687247s) [noise=true , address=104.36.82.39:4751 , piece_id=ZM64XI6VX7YENY6WSGRMS34XAATUXGEVWPISJMYOXXQB5LLSBJYA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128StmvpG1KYRXeu3gzEEfRmYZgR5ACJRf2xGZcEiZHsfk1KNyr): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245815813 (13h46m7.676895944s) [seg_pos={0 0}, upload=14368]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:12:13.245901858 (13h46m7.676981979s) [upload=14368]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:12:13.245907359 (13h46m7.67698749s) [upload=14368]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:12:13.789706183 (13h46m8.220786304s) [upload=14368]: Done uploading segments... Aug 9 07:12:13.789724909 (13h46m8.22080503s) [upload=14368]: Upload finished. err: <nil> Duration: 5.521410688s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................X..X..X......X.......X..........] duration: 5.99113578s FAIL (2025-08-09 07:28:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.991159145s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X....................X.....................] duration: 5.578137484s FAIL (2025-08-09 07:27:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after e3ff044d6ae4076192b1e1fe513b8dabe6c767a5c3203dea27eaa2ec31688bdd != before 663c2bc3be3b2c93bc26181187625e6a4f7893b1381782a3fe391ac4fca8ac68 ======LOGS====== Aug 9 07:12:10.752933843 (13h46m5.184013974s) [upload=14368]: Starting upload Aug 9 07:12:10.752992164 (13h46m5.184072285s) [upload=14368]: Done starting upload Aug 9 07:12:10.753168141 (13h46m5.184248273s) [upload=14368]: Uploading segments... Aug 9 07:12:10.753297318 (13h46m5.184377449s) [upload=14368]: Got next segment. Inline: false Aug 9 07:12:10.753326153 (13h46m5.184406274s) [upload=14368]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:12:10.753330122 (13h46m5.184410253s) [upload=14368]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 07:12:11.207655625 (13h46m5.638735756s) [seg_pos={0 0}, upload=14368]: Begin upload segment... Aug 9 07:12:11.207720700 (13h46m5.638800831s) [seg_pos={0 0}, upload=14368]: Done begin upload segment. Aug 9 07:12:11.207743194 (13h46m5.638823325s) [noise=true , address=104.36.82.44:4896 , piece_id=OHFYRMATCJBA2K3HRJ77ZJTOJ3HT7MK7N5JC7NZ52PXEJ35JKWHQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.207765646 (13h46m5.638845767s) [noise=true , address=64.49.13.27:4961 , piece_id=GYEU7TP4OXYEYFXENUVQRUA4B6Q75NU52NLSADC2AQ3477QHTBSA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208227149 (13h46m5.63930728s) [noise=true , address=64.49.13.139:4479 , piece_id=5IUHFINOLUATEGGJW3XJL4KPTUV2NC4YWICUUKHFEKBHSMRIA2RA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208364321 (13h46m5.639444463s) [noise=true , address=104.36.82.60:4370 , piece_id=T3P6UIMSI73BKLVUYXHWEKOXTEXWVOGFSG2NSVJ3GK6M2LBXSKZA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.208698120 (13h46m5.639778251s) [noise=true , address=104.36.82.8:4981 , piece_id=FOIG62XE46FFMD5UHH2SN6SX7KGFEV5YK72DM6WBFROHGC665ZPA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209043852 (13h46m5.640123983s) [noise=true , address=104.36.82.59:4324 , piece_id=ZWHTHIGUKE4BGL7XQGQ7SSQOKV2AS7KCKRU3IG7XOPN75FJCCIRA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209320301 (13h46m5.640400432s) [noise=true , address=64.49.13.169:4715 , piece_id=F4XFTIMZVJQFZ7WPSM65CRORDOAJ5RXNFD2LW7JWRUISIB2OQQTQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.209683986 (13h46m5.640764117s) [noise=true , address=64.49.13.170:4724 , piece_id=NGADM4QFZPZV2ZQ3T7D55CW5YASOZA6NCPBA2RECAJQ543YBBXHA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210169786 (13h46m5.641249917s) [noise=true , address=104.36.82.60:4371 , piece_id=7A4ZKM7IMWTIEPXVP4VWYWXEZZ322WJPV7FNFDHS67OFVQK4QASA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210381952 (13h46m5.641462083s) [noise=true , address=104.36.82.61:4404 , piece_id=42LT6BXLMKSGA6Q7P3IR4SZZZLAMKUHJO4QWZYULTV5AFMCHPL3Q, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210636339 (13h46m5.64171647s) [noise=true , address=104.36.82.43:4847 , piece_id=5KKBDZL65I2KSGRPO3OHVV4UBY7XVV2NLTTKOY2BP65ESCKFSSRA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.210856220 (13h46m5.641936351s) [noise=true , address=104.36.82.16:4316 , piece_id=4NPL3A5HFVERECAHTLPRIGBWPFA7QW4L2WTX55W7L2DCYUEG75XA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211079006 (13h46m5.642159127s) [noise=true , address=104.36.82.14:4237 , piece_id=RLIE5EYM6A7VNRDTTFXJTUV7WJKJ5QDKZEAOMUZYA3QFNWXPQKNQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211304737 (13h46m5.642384868s) [noise=true , address=45.91.116.7:4031 , piece_id=I3M2I7TE2HH6KKY4R7VB5PSNZJXAVCK36UULZ2BLWNKWKTYTJIQQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211491064 (13h46m5.642571195s) [noise=true , address=104.36.82.37:4669 , piece_id=KK2GNVWD6EUIBG77ZIEAH3ZBDDXO4PX2DFCFPWDTCT4KFTG5ESLA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.211981352 (13h46m5.643061483s) [noise=true , address=104.36.82.15:4292 , piece_id=3FMM25ACGF4RPBO66JM4PMB2BC2DCVGCR3JCMWCONRBYSODONGFA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.212224658 (13h46m5.643304789s) [noise=true , address=104.36.82.3:4795 , piece_id=DDRQ7JUPFC5FTKGOT6J4YHIXJXEGPOVQYPV3EQ35RDGHVU3H4OUQ, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:11.212477902 (13h46m5.643558033s) [noise=true , address=104.36.82.39:4751 , piece_id=ZM64XI6VX7YENY6WSGRMS34XAATUXGEVWPISJMYOXXQB5LLSBJYA, seg_pos={0 0}, upload=14368]: Uploading piece... Aug 9 07:12:12.328114668 (13h46m6.759194799s) [upload=14368]: Next returned nil segment Aug 9 07:12:12.328130238 (13h46m6.759210359s) [upload=14368]: Waiting for error group managing segments... Aug 9 07:12:12.615123755 (13h46m7.046203886s) [noise=true , address=64.49.13.170:4724 , piece_id=NGADM4QFZPZV2ZQ3T7D55CW5YASOZA6NCPBA2RECAJQ543YBBXHA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.633849403 (13h46m7.064929534s) [noise=true , address=64.49.13.139:4479 , piece_id=5IUHFINOLUATEGGJW3XJL4KPTUV2NC4YWICUUKHFEKBHSMRIA2RA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.637449872 (13h46m7.068530003s) [noise=true , address=64.49.13.27:4961 , piece_id=GYEU7TP4OXYEYFXENUVQRUA4B6Q75NU52NLSADC2AQ3477QHTBSA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:12.650403255 (13h46m7.081483387s) [noise=true , address=64.49.13.169:4715 , piece_id=F4XFTIMZVJQFZ7WPSM65CRORDOAJ5RXNFD2LW7JWRUISIB2OQQTQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.137896401 (13h46m7.568976532s) [noise=true , address=104.36.82.60:4371 , piece_id=7A4ZKM7IMWTIEPXVP4VWYWXEZZ322WJPV7FNFDHS67OFVQK4QASA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.139324974 (13h46m7.570405095s) [noise=true , address=104.36.82.8:4981 , piece_id=FOIG62XE46FFMD5UHH2SN6SX7KGFEV5YK72DM6WBFROHGC665ZPA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.146401663 (13h46m7.577481794s) [noise=true , address=45.91.116.7:4031 , piece_id=I3M2I7TE2HH6KKY4R7VB5PSNZJXAVCK36UULZ2BLWNKWKTYTJIQQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.161160482 (13h46m7.592240603s) [noise=true , address=104.36.82.43:4847 , piece_id=5KKBDZL65I2KSGRPO3OHVV4UBY7XVV2NLTTKOY2BP65ESCKFSSRA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.165737920 (13h46m7.596818041s) [noise=true , address=104.36.82.15:4292 , piece_id=3FMM25ACGF4RPBO66JM4PMB2BC2DCVGCR3JCMWCONRBYSODONGFA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.168998608 (13h46m7.600078729s) [noise=true , address=104.36.82.60:4370 , piece_id=T3P6UIMSI73BKLVUYXHWEKOXTEXWVOGFSG2NSVJ3GK6M2LBXSKZA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.209005594 (13h46m7.640085725s) [noise=true , address=104.36.82.59:4324 , piece_id=ZWHTHIGUKE4BGL7XQGQ7SSQOKV2AS7KCKRU3IG7XOPN75FJCCIRA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.229451654 (13h46m7.660531785s) [noise=true , address=104.36.82.3:4795 , piece_id=DDRQ7JUPFC5FTKGOT6J4YHIXJXEGPOVQYPV3EQ35RDGHVU3H4OUQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.238304101 (13h46m7.669384232s) [noise=true , address=104.36.82.61:4404 , piece_id=42LT6BXLMKSGA6Q7P3IR4SZZZLAMKUHJO4QWZYULTV5AFMCHPL3Q, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.245475733 (13h46m7.676555854s) [noise=true , address=104.36.82.37:4669 , piece_id=KK2GNVWD6EUIBG77ZIEAH3ZBDDXO4PX2DFCFPWDTCT4KFTG5ESLA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: <nil> Aug 9 07:12:13.245503065 (13h46m7.676583196s) [seg_pos={0 0}, upload=14368]: Segment reached optimal threshold of 14 pieces. Aug 9 07:12:13.245606703 (13h46m7.676686834s) [noise=true , address=104.36.82.14:4237 , piece_id=RLIE5EYM6A7VNRDTTFXJTUV7WJKJ5QDKZEAOMUZYA3QFNWXPQKNQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245678009 (13h46m7.67675814s) [noise=true , address=104.36.82.16:4316 , piece_id=4NPL3A5HFVERECAHTLPRIGBWPFA7QW4L2WTX55W7L2DCYUEG75XA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PPnXDzwXDVDKemw483GXuzQrPN48JJrjeeD7az2NPNawHNaXf): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245759245 (13h46m7.676839366s) [noise=true , address=104.36.82.44:4896 , piece_id=OHFYRMATCJBA2K3HRJ77ZJTOJ3HT7MK7N5JC7NZ52PXEJ35JKWHQ, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ndSgjgfZMVQni5tw1ugdTrhJt4c4NagdvgYTbSbjatYYtfu1o): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245792349 (13h46m7.67687247s) [noise=true , address=104.36.82.39:4751 , piece_id=ZM64XI6VX7YENY6WSGRMS34XAATUXGEVWPISJMYOXXQB5LLSBJYA, seg_pos={0 0}, upload=14368]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128StmvpG1KYRXeu3gzEEfRmYZgR5ACJRf2xGZcEiZHsfk1KNyr): protocol: expected piece hash; context canceled; EOF Aug 9 07:12:13.245815813 (13h46m7.676895944s) [seg_pos={0 0}, upload=14368]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 07:12:13.245901858 (13h46m7.676981979s) [upload=14368]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 07:12:13.245907359 (13h46m7.67698749s) [upload=14368]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:12:13.789706183 (13h46m8.220786304s) [upload=14368]: Done uploading segments... Aug 9 07:12:13.789724909 (13h46m8.22080503s) [upload=14368]: Upload finished. err: <nil> Duration: 5.578158192s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X..........X............] duration: 1.862699367s FAIL (2025-08-09 07:27:42 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 07:19:01.525821703 (13h52m55.956901834s) [upload=14491]: Starting upload Aug 9 07:19:01.525874865 (13h52m55.956954996s) [upload=14491]: Done starting upload Aug 9 07:19:01.525947474 (13h52m55.957027605s) [upload=14491]: Uploading segments... Aug 9 07:19:01.526043047 (13h52m55.957123178s) [upload=14491]: Got next segment. Inline: true Aug 9 07:19:01.526048717 (13h52m55.957128848s) [upload=14491]: Waiting for error group managing segments... Aug 9 07:19:01.526106067 (13h52m55.957186198s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 07:19:01.526111218 (13h52m55.957191349s) [upload=14491]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 07:19:01.526114113 (13h52m55.957194244s) [upload=14491]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 07:19:01.792255834 (13h52m56.223335965s) [upload=14491]: Done uploading segments... Aug 9 07:19:01.792276393 (13h52m56.223356524s) [upload=14491]: Upload finished. err: <nil> Duration:* 1.862769961s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................X..X..X......X.......X.........] duration: 6.445841961s FAIL (2025-08-09 07:27:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.445891265s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)