Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.756378909s FAIL (2025-07-16 10:27:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.756405852s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.753556785s FAIL (2025-07-16 10:26:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.753583005s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.716615505s FAIL (2025-07-16 10:26:41 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.716644452s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.759373088s FAIL (2025-07-16 10:26:05 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.759463692s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.773262173s FAIL (2025-07-16 10:25:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.773282033s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.814195112s FAIL (2025-07-16 10:24:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.814218547s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.717361767s FAIL (2025-07-16 10:23:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.717383649s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.775950828s FAIL (2025-07-16 10:21:47 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.775976768s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.76783052s FAIL (2025-07-16 10:21:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.767866569s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.74556585s FAIL (2025-07-16 10:20:32 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.745591259s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.751470381s FAIL (2025-07-16 10:20:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.751498503s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.87443518s FAIL (2025-07-16 10:19:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.874475278s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.680764343s FAIL (2025-07-16 10:18:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.680794571s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.821828448s FAIL (2025-07-16 10:17:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.821846873s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.773859497s FAIL (2025-07-16 10:16:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.773881849s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.691529741s FAIL (2025-07-16 10:16:02 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.69155496s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.434631356s FAIL (2025-07-16 10:15:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 6.43465919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.736947734s FAIL (2025-07-16 10:14:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.736970939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.82803029s FAIL (2025-07-16 10:13:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.828056129s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.723147656s FAIL (2025-07-16 10:12:52 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.723175159s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.774165853s FAIL (2025-07-16 10:12:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.774194699s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.712015587s FAIL (2025-07-16 10:11:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.712048631s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.665388643s FAIL (2025-07-16 10:10:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.665411416s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.666862737s FAIL (2025-07-16 10:09:43 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.666942569s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.862477433s FAIL (2025-07-16 10:09:00 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.862505355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.840178325s FAIL (2025-07-16 10:07:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.840221057s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.720477993s FAIL (2025-07-16 10:07:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.720502271s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.74139728s FAIL (2025-07-16 10:06:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.741419363s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.931037471s FAIL (2025-07-16 10:06:05 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.931064473s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.70189701s FAIL (2025-07-16 10:05:41 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.701924654s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.954323015s FAIL (2025-07-16 10:04:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.954349757s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.420315449s FAIL (2025-07-16 10:03:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 6.420343873s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.814724525s FAIL (2025-07-16 10:02:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.814753732s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.789877986s FAIL (2025-07-16 10:02:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 5.789909858s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................................X] duration: 30.000470713s FAIL (2025-07-16 10:02:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 247393737 (552h49m37.018035092s) [noise=true , address=45.91.116.5:4298 , piece_id=SO5TK54HNRJYPBWWO7G5Z7LILWBVYRO7GYSUBLGXP3AENSNNWUHQ, seg_pos={0 0}, upload=442102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:02.247425508 (552h49m37.018066853s) [seg_pos={0 0}, upload=442102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:02.247519749 (552h49m37.018161094s) [upload=442102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:02.247526431 (552h49m37.018167776s) [upload=442102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:02.567193618 (552h49m37.337834964s) [upload=442102]: Done uploading segments... Jul 16 10:00:02.567223626 (552h49m37.337864971s) [upload=442102]: Upload finished. err: <nil> Jul 16 10:00:35.322798409 (552h50m10.093439744s) [upload=442109]: Starting upload Jul 16 10:00:35.322854246 (552h50m10.093495581s) [upload=442109]: Done starting upload Jul 16 10:00:35.322959147 (552h50m10.093600482s) [upload=442109]: Uploading segments... Jul 16 10:00:35.323170992 (552h50m10.093812327s) [upload=442109]: Got next segment. Inline: false Jul 16 10:00:35.323212882 (552h50m10.093854217s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 10:00:35.323218092 (552h50m10.093859427s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 10:00:35.563095085 (552h50m10.33373642s) [seg_pos={0 0}, upload=442109]: Begin upload segment... Jul 16 10:00:35.563136655 (552h50m10.33377799s) [seg_pos={0 0}, upload=442109]: Done begin upload segment. Jul 16 10:00:35.563158777 (552h50m10.333800112s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563198944 (552h50m10.333840279s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563587467 (552h50m10.334228802s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563707898 (552h50m10.334349233s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.563853246 (552h50m10.334494581s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564066364 (552h50m10.334707709s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564126779 (552h50m10.334768114s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564460948 (552h50m10.335102283s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564516885 (552h50m10.33515822s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564768236 (552h50m10.335409571s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.564775581 (552h50m10.335416906s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565144586 (552h50m10.335785922s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.565412429 (552h50m10.336053794s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.566888023 (552h50m10.337529358s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.567630944 (552h50m10.338272279s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:35.569125584 (552h50m10.339766919s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Uploading piece... Jul 16 10:00:37.312772479 (552h50m12.083413814s) [upload=442109]: Next returned nil segment Jul 16 10:00:37.312790493 (552h50m12.083431828s) [upload=442109]: Waiting for error group managing segments... Jul 16 10:00:37.621481541 (552h50m12.392122876s) [noise=true , address=45.91.116.14:4189 , piece_id=2EOPVJYRHRLZE57BPRX6AO7GAFSNOS5GDJ3ARZAICQUNTAMF3JOA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.622538452 (552h50m12.393179787s) [noise=true , address=45.91.116.8:4047 , piece_id=PZOXUWLARQPBNSN67OX2ANYWWWOUXZ4BJTQAQRRDX3YWR7LUC7JQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625336787 (552h50m12.395978122s) [noise=true , address=45.91.116.11:4118 , piece_id=6AJYPKWRE27Y5VYRHV643PH3MWLR7DQ3CJG7CRNGFUU5DXIPP6LA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.625486202 (552h50m12.396127537s) [noise=true , address=45.91.116.13:4154 , piece_id=7T7C7GQK6MMN2WB5K5GMT7MVIS5LDE7ZVJ3NJXUYSBROTVWED5QA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.643097378 (552h50m12.413738723s) [noise=true , address=45.91.116.12:4227 , piece_id=BUY5Z3M7E6QDKLHNIODRQK3RKJ32XPNXZG7EDKQF6IWFBRAPC67Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.646933618 (552h50m12.417574953s) [noise=true , address=77.90.31.11:4553 , piece_id=ER33GQYXW5WQCSR7MTAFCS5LREVU4IEQFZVU25MIK27TOVZJSPMQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.656912601 (552h50m12.427553936s) [noise=true , address=77.90.31.14:4661 , piece_id=RQUAB4AGZSNROLUFJEVE2F4YO67LJDYQKWQ4KXDZDUACMAZ36ZXA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.657051858 (552h50m12.427693193s) [noise=true , address=77.90.31.16:4735 , piece_id=HD3S7QZXO4RMPYCL23T3Z74YTJVG7C5MYBWKB244J7WKDNNXCCTA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.658382904 (552h50m12.429024239s) [noise=true , address=77.90.31.18:4806 , piece_id=XUQS52FOQ3QI3V3VXZ2T3OJIML4TFADNTK4IUR4QZMHXOFOGXI7A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.659112881 (552h50m12.429754216s) [noise=true , address=77.90.31.12:4589 , piece_id=PP4Y5NRJ6JVK4LE6BGTRDN4MCD5JBUY7REZ4BBVQ65YUVZ7UCHWA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.662495022 (552h50m12.433136357s) [noise=true , address=77.90.31.10:4519 , piece_id=Y7FCGTWSZUYYQVVN4A2K5Y6HXYOYUBTDZ4BB3GKZ3CG36M4JG72Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.663463916 (552h50m12.434105251s) [noise=true , address=77.90.31.13:4628 , piece_id=JFF7FVEJF4PI3EDCTMGCRWKRT536IKPR367EJDSU6NL5K432I2MQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695800476 (552h50m12.466441811s) [noise=true , address=45.91.116.10:4082 , piece_id=5SQCN7SRWSQWDEUD6CWGKXPIRJZ3AUTMRE3TWOSIE5LB6AHXI56Q, seg_pos={0 0}, upload=442109]: Done uploading piece. err: <nil> Jul 16 10:00:37.695821076 (552h50m12.466462411s) [seg_pos={0 0}, upload=442109]: Segment reached optimal threshold of 13 pieces. Jul 16 10:00:37.695931788 (552h50m12.466573133s) [noise=true , address=104.36.82.9:4005 , piece_id=VRB6S4DFGLIOXJUK3HODA7F3MNEGWCJKLEQ75TUFEPZW334UITVQ, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.695964340 (552h50m12.466605675s) [noise=true , address=45.91.116.7:4011 , piece_id=FO6J6XC73ANPA23IMGP3D5K5MJVNTX7EBJT3ASYUWAJTSK3PLS2A, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696027892 (552h50m12.466669227s) [noise=true , address=104.36.82.3:4767 , piece_id=CCBSXDZKI6GJNPKY5N46KQOMOJQU6YGFH3NTRYKIGZJOCGV3FEBA, seg_pos={0 0}, upload=442109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Jul 16 10:00:37.696084500 (552h50m12.466725835s) [seg_pos={0 0}, upload=442109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 10:00:37.696142571 (552h50m12.466783896s) [upload=442109]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 10:00:37.696146709 (552h50m12.466788044s) [upload=442109]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 10:00:38.117285014 (552h50m12.887926599s) [upload=442109]: Done uploading segments... Jul 16 10:00:38.117304631 (552h50m12.887945967s) [upload=442109]: Upload finished. err: <nil> Duration: 30.000493227s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.29975083s FAIL (2025-07-16 10:01:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.299786137s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.597888856s FAIL (2025-07-16 10:01:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.597915147s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.826602141s FAIL (2025-07-16 10:00:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== [upload=441499]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 09:14:11.682971031 (552h3m46.453612367s) [upload=441499]: Done uploading segments... Jul 16 09:14:11.682988174 (552h3m46.453629509s) [upload=441499]: Upload finished. err: <nil> Jul 16 09:14:26.327645611 (552h4m1.098286946s) [upload=441503]: Starting upload Jul 16 09:14:26.327745291 (552h4m1.098386626s) [upload=441503]: Done starting upload Jul 16 09:14:26.327841095 (552h4m1.09848243s) [upload=441503]: Uploading segments... Jul 16 09:14:26.328174823 (552h4m1.098816178s) [upload=441503]: Got next segment. Inline: false Jul 16 09:14:26.328215471 (552h4m1.098856806s) [upload=441503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 09:14:26.328220200 (552h4m1.098861535s) [upload=441503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 09:14:26.564728127 (552h4m1.335369462s) [seg_pos={0 0}, upload=441503]: Begin upload segment... Jul 16 09:14:26.564766972 (552h4m1.335408307s) [seg_pos={0 0}, upload=441503]: Done begin upload segment. Jul 16 09:14:26.564794715 (552h4m1.33543605s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.564803291 (552h4m1.335444626s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565020106 (552h4m1.335661441s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565136919 (552h4m1.335778254s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565295302 (552h4m1.335936637s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565394593 (552h4m1.336035928s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565541323 (552h4m1.336182658s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565650973 (552h4m1.336292308s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565769309 (552h4m1.336410644s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565947050 (552h4m1.336588385s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566007525 (552h4m1.33664886s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566204452 (552h4m1.336845777s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566224230 (552h4m1.336865555s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566460381 (552h4m1.337101716s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566660565 (552h4m1.3373019s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566854426 (552h4m1.337495761s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:28.257214459 (552h4m3.027855784s) [upload=441503]: Next returned nil segment Jul 16 09:14:28.257230961 (552h4m3.027872296s) [upload=441503]: Waiting for error group managing segments... Jul 16 09:14:28.607934261 (552h4m3.378575596s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.629612098 (552h4m3.400253433s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.631792701 (552h4m3.402434036s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.632101531 (552h4m3.402742867s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696100141 (552h4m3.466741477s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696305525 (552h4m3.46694685s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.707107683 (552h4m3.477749018s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.708545894 (552h4m3.479187219s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.711016491 (552h4m3.481657816s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.774412499 (552h4m3.545053834s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:42.485153717 (552h4m17.255795052s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:54.799737178 (552h4m29.570378513s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:55.680653470 (552h4m30.451294805s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection Jul 16 09:14:55.680714697 (552h4m30.451356032s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection Jul 16 09:14:55.680756227 (552h4m30.451397562s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection Jul 16 09:14:55.680828745 (552h4m30.45147008s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection Jul 16 09:14:55.680871197 (552h4m30.451512532s) [seg_pos={0 0}, upload=441503]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 09:14:55.927916416 (552h4m30.698557751s) [upload=441503]: Done uploading segments... Jul 16 09:14:55.927940001 (552h4m30.698581336s) [upload=441503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.826629794s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.294684397s FAIL (2025-07-16 10:00:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.294707721s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.762484656s FAIL (2025-07-16 10:00:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== [upload=441499]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 09:14:11.682971031 (552h3m46.453612367s) [upload=441499]: Done uploading segments... Jul 16 09:14:11.682988174 (552h3m46.453629509s) [upload=441499]: Upload finished. err: <nil> Jul 16 09:14:26.327645611 (552h4m1.098286946s) [upload=441503]: Starting upload Jul 16 09:14:26.327745291 (552h4m1.098386626s) [upload=441503]: Done starting upload Jul 16 09:14:26.327841095 (552h4m1.09848243s) [upload=441503]: Uploading segments... Jul 16 09:14:26.328174823 (552h4m1.098816178s) [upload=441503]: Got next segment. Inline: false Jul 16 09:14:26.328215471 (552h4m1.098856806s) [upload=441503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 09:14:26.328220200 (552h4m1.098861535s) [upload=441503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 09:14:26.564728127 (552h4m1.335369462s) [seg_pos={0 0}, upload=441503]: Begin upload segment... Jul 16 09:14:26.564766972 (552h4m1.335408307s) [seg_pos={0 0}, upload=441503]: Done begin upload segment. Jul 16 09:14:26.564794715 (552h4m1.33543605s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.564803291 (552h4m1.335444626s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565020106 (552h4m1.335661441s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565136919 (552h4m1.335778254s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565295302 (552h4m1.335936637s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565394593 (552h4m1.336035928s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565541323 (552h4m1.336182658s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565650973 (552h4m1.336292308s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565769309 (552h4m1.336410644s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565947050 (552h4m1.336588385s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566007525 (552h4m1.33664886s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566204452 (552h4m1.336845777s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566224230 (552h4m1.336865555s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566460381 (552h4m1.337101716s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566660565 (552h4m1.3373019s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566854426 (552h4m1.337495761s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:28.257214459 (552h4m3.027855784s) [upload=441503]: Next returned nil segment Jul 16 09:14:28.257230961 (552h4m3.027872296s) [upload=441503]: Waiting for error group managing segments... Jul 16 09:14:28.607934261 (552h4m3.378575596s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.629612098 (552h4m3.400253433s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.631792701 (552h4m3.402434036s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.632101531 (552h4m3.402742867s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696100141 (552h4m3.466741477s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696305525 (552h4m3.46694685s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.707107683 (552h4m3.477749018s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.708545894 (552h4m3.479187219s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.711016491 (552h4m3.481657816s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.774412499 (552h4m3.545053834s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:42.485153717 (552h4m17.255795052s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:54.799737178 (552h4m29.570378513s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:55.680653470 (552h4m30.451294805s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection Jul 16 09:14:55.680714697 (552h4m30.451356032s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection Jul 16 09:14:55.680756227 (552h4m30.451397562s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection Jul 16 09:14:55.680828745 (552h4m30.45147008s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection Jul 16 09:14:55.680871197 (552h4m30.451512532s) [seg_pos={0 0}, upload=441503]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 09:14:55.927916416 (552h4m30.698557751s) [upload=441503]: Done uploading segments... Jul 16 09:14:55.927940001 (552h4m30.698581336s) [upload=441503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.762509745s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.889761175s FAIL (2025-07-16 09:59:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.889782827s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X...X.......................................................] duration: 6.43821574s FAIL (2025-07-16 09:59:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.438244465s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X...X......................................................] duration: 6.12541972s FAIL (2025-07-16 09:59:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.125439218s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.765021516s FAIL (2025-07-16 09:58:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== [upload=441499]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 09:14:11.682971031 (552h3m46.453612367s) [upload=441499]: Done uploading segments... Jul 16 09:14:11.682988174 (552h3m46.453629509s) [upload=441499]: Upload finished. err: <nil> Jul 16 09:14:26.327645611 (552h4m1.098286946s) [upload=441503]: Starting upload Jul 16 09:14:26.327745291 (552h4m1.098386626s) [upload=441503]: Done starting upload Jul 16 09:14:26.327841095 (552h4m1.09848243s) [upload=441503]: Uploading segments... Jul 16 09:14:26.328174823 (552h4m1.098816178s) [upload=441503]: Got next segment. Inline: false Jul 16 09:14:26.328215471 (552h4m1.098856806s) [upload=441503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 09:14:26.328220200 (552h4m1.098861535s) [upload=441503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 09:14:26.564728127 (552h4m1.335369462s) [seg_pos={0 0}, upload=441503]: Begin upload segment... Jul 16 09:14:26.564766972 (552h4m1.335408307s) [seg_pos={0 0}, upload=441503]: Done begin upload segment. Jul 16 09:14:26.564794715 (552h4m1.33543605s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.564803291 (552h4m1.335444626s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565020106 (552h4m1.335661441s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565136919 (552h4m1.335778254s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565295302 (552h4m1.335936637s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565394593 (552h4m1.336035928s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565541323 (552h4m1.336182658s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565650973 (552h4m1.336292308s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565769309 (552h4m1.336410644s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565947050 (552h4m1.336588385s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566007525 (552h4m1.33664886s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566204452 (552h4m1.336845777s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566224230 (552h4m1.336865555s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566460381 (552h4m1.337101716s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566660565 (552h4m1.3373019s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566854426 (552h4m1.337495761s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:28.257214459 (552h4m3.027855784s) [upload=441503]: Next returned nil segment Jul 16 09:14:28.257230961 (552h4m3.027872296s) [upload=441503]: Waiting for error group managing segments... Jul 16 09:14:28.607934261 (552h4m3.378575596s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.629612098 (552h4m3.400253433s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.631792701 (552h4m3.402434036s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.632101531 (552h4m3.402742867s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696100141 (552h4m3.466741477s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696305525 (552h4m3.46694685s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.707107683 (552h4m3.477749018s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.708545894 (552h4m3.479187219s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.711016491 (552h4m3.481657816s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.774412499 (552h4m3.545053834s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:42.485153717 (552h4m17.255795052s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:54.799737178 (552h4m29.570378513s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:55.680653470 (552h4m30.451294805s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection Jul 16 09:14:55.680714697 (552h4m30.451356032s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection Jul 16 09:14:55.680756227 (552h4m30.451397562s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection Jul 16 09:14:55.680828745 (552h4m30.45147008s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection Jul 16 09:14:55.680871197 (552h4m30.451512532s) [seg_pos={0 0}, upload=441503]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 09:14:55.927916416 (552h4m30.698557751s) [upload=441503]: Done uploading segments... Jul 16 09:14:55.927940001 (552h4m30.698581336s) [upload=441503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.765057586s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.74066375s FAIL (2025-07-16 09:58:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== [upload=441499]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 09:14:11.682971031 (552h3m46.453612367s) [upload=441499]: Done uploading segments... Jul 16 09:14:11.682988174 (552h3m46.453629509s) [upload=441499]: Upload finished. err: <nil> Jul 16 09:14:26.327645611 (552h4m1.098286946s) [upload=441503]: Starting upload Jul 16 09:14:26.327745291 (552h4m1.098386626s) [upload=441503]: Done starting upload Jul 16 09:14:26.327841095 (552h4m1.09848243s) [upload=441503]: Uploading segments... Jul 16 09:14:26.328174823 (552h4m1.098816178s) [upload=441503]: Got next segment. Inline: false Jul 16 09:14:26.328215471 (552h4m1.098856806s) [upload=441503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 09:14:26.328220200 (552h4m1.098861535s) [upload=441503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 09:14:26.564728127 (552h4m1.335369462s) [seg_pos={0 0}, upload=441503]: Begin upload segment... Jul 16 09:14:26.564766972 (552h4m1.335408307s) [seg_pos={0 0}, upload=441503]: Done begin upload segment. Jul 16 09:14:26.564794715 (552h4m1.33543605s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.564803291 (552h4m1.335444626s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565020106 (552h4m1.335661441s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565136919 (552h4m1.335778254s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565295302 (552h4m1.335936637s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565394593 (552h4m1.336035928s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565541323 (552h4m1.336182658s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565650973 (552h4m1.336292308s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565769309 (552h4m1.336410644s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565947050 (552h4m1.336588385s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566007525 (552h4m1.33664886s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566204452 (552h4m1.336845777s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566224230 (552h4m1.336865555s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566460381 (552h4m1.337101716s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566660565 (552h4m1.3373019s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566854426 (552h4m1.337495761s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:28.257214459 (552h4m3.027855784s) [upload=441503]: Next returned nil segment Jul 16 09:14:28.257230961 (552h4m3.027872296s) [upload=441503]: Waiting for error group managing segments... Jul 16 09:14:28.607934261 (552h4m3.378575596s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.629612098 (552h4m3.400253433s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.631792701 (552h4m3.402434036s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.632101531 (552h4m3.402742867s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696100141 (552h4m3.466741477s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696305525 (552h4m3.46694685s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.707107683 (552h4m3.477749018s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.708545894 (552h4m3.479187219s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.711016491 (552h4m3.481657816s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.774412499 (552h4m3.545053834s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:42.485153717 (552h4m17.255795052s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:54.799737178 (552h4m29.570378513s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:55.680653470 (552h4m30.451294805s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection Jul 16 09:14:55.680714697 (552h4m30.451356032s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection Jul 16 09:14:55.680756227 (552h4m30.451397562s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection Jul 16 09:14:55.680828745 (552h4m30.45147008s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection Jul 16 09:14:55.680871197 (552h4m30.451512532s) [seg_pos={0 0}, upload=441503]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 09:14:55.927916416 (552h4m30.698557751s) [upload=441503]: Done uploading segments... Jul 16 09:14:55.927940001 (552h4m30.698581336s) [upload=441503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.740739245s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X...X.....................................................] duration: 6.307355169s FAIL (2025-07-16 09:58:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.307384043s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X...X....................................................] duration: 6.30740567s FAIL (2025-07-16 09:57:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.307435017s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.733434685s FAIL (2025-07-16 09:57:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== [upload=441499]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 09:14:11.682971031 (552h3m46.453612367s) [upload=441499]: Done uploading segments... Jul 16 09:14:11.682988174 (552h3m46.453629509s) [upload=441499]: Upload finished. err: <nil> Jul 16 09:14:26.327645611 (552h4m1.098286946s) [upload=441503]: Starting upload Jul 16 09:14:26.327745291 (552h4m1.098386626s) [upload=441503]: Done starting upload Jul 16 09:14:26.327841095 (552h4m1.09848243s) [upload=441503]: Uploading segments... Jul 16 09:14:26.328174823 (552h4m1.098816178s) [upload=441503]: Got next segment. Inline: false Jul 16 09:14:26.328215471 (552h4m1.098856806s) [upload=441503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 09:14:26.328220200 (552h4m1.098861535s) [upload=441503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 09:14:26.564728127 (552h4m1.335369462s) [seg_pos={0 0}, upload=441503]: Begin upload segment... Jul 16 09:14:26.564766972 (552h4m1.335408307s) [seg_pos={0 0}, upload=441503]: Done begin upload segment. Jul 16 09:14:26.564794715 (552h4m1.33543605s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.564803291 (552h4m1.335444626s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565020106 (552h4m1.335661441s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565136919 (552h4m1.335778254s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565295302 (552h4m1.335936637s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565394593 (552h4m1.336035928s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565541323 (552h4m1.336182658s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565650973 (552h4m1.336292308s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565769309 (552h4m1.336410644s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565947050 (552h4m1.336588385s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566007525 (552h4m1.33664886s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566204452 (552h4m1.336845777s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566224230 (552h4m1.336865555s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566460381 (552h4m1.337101716s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566660565 (552h4m1.3373019s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566854426 (552h4m1.337495761s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:28.257214459 (552h4m3.027855784s) [upload=441503]: Next returned nil segment Jul 16 09:14:28.257230961 (552h4m3.027872296s) [upload=441503]: Waiting for error group managing segments... Jul 16 09:14:28.607934261 (552h4m3.378575596s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.629612098 (552h4m3.400253433s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.631792701 (552h4m3.402434036s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.632101531 (552h4m3.402742867s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696100141 (552h4m3.466741477s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696305525 (552h4m3.46694685s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.707107683 (552h4m3.477749018s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.708545894 (552h4m3.479187219s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.711016491 (552h4m3.481657816s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.774412499 (552h4m3.545053834s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:42.485153717 (552h4m17.255795052s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:54.799737178 (552h4m29.570378513s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:55.680653470 (552h4m30.451294805s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection Jul 16 09:14:55.680714697 (552h4m30.451356032s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection Jul 16 09:14:55.680756227 (552h4m30.451397562s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection Jul 16 09:14:55.680828745 (552h4m30.45147008s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection Jul 16 09:14:55.680871197 (552h4m30.451512532s) [seg_pos={0 0}, upload=441503]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 09:14:55.927916416 (552h4m30.698557751s) [upload=441503]: Done uploading segments... Jul 16 09:14:55.927940001 (552h4m30.698581336s) [upload=441503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.733456096s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X...X...................................................] duration: 6.467043327s FAIL (2025-07-16 09:56:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.467069886s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.721257276s FAIL (2025-07-16 09:56:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== [upload=441499]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 09:14:11.682971031 (552h3m46.453612367s) [upload=441499]: Done uploading segments... Jul 16 09:14:11.682988174 (552h3m46.453629509s) [upload=441499]: Upload finished. err: <nil> Jul 16 09:14:26.327645611 (552h4m1.098286946s) [upload=441503]: Starting upload Jul 16 09:14:26.327745291 (552h4m1.098386626s) [upload=441503]: Done starting upload Jul 16 09:14:26.327841095 (552h4m1.09848243s) [upload=441503]: Uploading segments... Jul 16 09:14:26.328174823 (552h4m1.098816178s) [upload=441503]: Got next segment. Inline: false Jul 16 09:14:26.328215471 (552h4m1.098856806s) [upload=441503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 09:14:26.328220200 (552h4m1.098861535s) [upload=441503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 09:14:26.564728127 (552h4m1.335369462s) [seg_pos={0 0}, upload=441503]: Begin upload segment... Jul 16 09:14:26.564766972 (552h4m1.335408307s) [seg_pos={0 0}, upload=441503]: Done begin upload segment. Jul 16 09:14:26.564794715 (552h4m1.33543605s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.564803291 (552h4m1.335444626s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565020106 (552h4m1.335661441s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565136919 (552h4m1.335778254s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565295302 (552h4m1.335936637s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565394593 (552h4m1.336035928s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565541323 (552h4m1.336182658s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565650973 (552h4m1.336292308s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565769309 (552h4m1.336410644s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.565947050 (552h4m1.336588385s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566007525 (552h4m1.33664886s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566204452 (552h4m1.336845777s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566224230 (552h4m1.336865555s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566460381 (552h4m1.337101716s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566660565 (552h4m1.3373019s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:26.566854426 (552h4m1.337495761s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Uploading piece... Jul 16 09:14:28.257214459 (552h4m3.027855784s) [upload=441503]: Next returned nil segment Jul 16 09:14:28.257230961 (552h4m3.027872296s) [upload=441503]: Waiting for error group managing segments... Jul 16 09:14:28.607934261 (552h4m3.378575596s) [noise=true , address=45.91.116.10:4082 , piece_id=DSZSA7W3UKX6CWY6OGQSRTWRZY6RHFWXRMRQPTOFLQT6DTBDAD7Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.629612098 (552h4m3.400253433s) [noise=true , address=45.91.116.5:4297 , piece_id=7OANNLUSXNO5VM46DNUMVBKVBPQBO534EQXGRCKUAK5KPA4GWWIA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.631792701 (552h4m3.402434036s) [noise=true , address=45.91.116.9:4263 , piece_id=BJXA6KUUH4O46OPCE44QM7YMT7UWDFQSHZ57WIMGJW5ODB3YZ5YQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.632101531 (552h4m3.402742867s) [noise=true , address=45.91.116.11:4117 , piece_id=WDTV22X7MQOOZRFRBKVXWHSEJIL6UOTEI7DKHZG5CCA37JGRRIEQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696100141 (552h4m3.466741477s) [noise=true , address=45.91.116.4:4938 , piece_id=O6KYEQU2MS3HSHCYTX523PWESXWXIQMLMV26MW2UFSGFLMXGZIOA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.696305525 (552h4m3.46694685s) [noise=true , address=45.91.116.8:4045 , piece_id=B3JV4TBSUAPH4ML6EB7XPZKGKUVUEAMOMMCQOPGAP7A372FUSWDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.707107683 (552h4m3.477749018s) [noise=true , address=104.36.82.11:4113 , piece_id=MOV7NXAYT5QRM7E2R6ZDRPEHQ4TPOSVS3ST4Q2P5ERQSKQKQKYIQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.708545894 (552h4m3.479187219s) [noise=true , address=45.91.116.6:4975 , piece_id=FUUBJYUMOYNEAO3NCFQLK5YB6IINFRUKGYJF5KWWQODZ5ZMEND5A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.711016491 (552h4m3.481657816s) [noise=true , address=104.36.82.3:4767 , piece_id=XHOFSJNPVJOW5FB27UAMJNXVFCIDO342QUXUUPPL7BA2SO7M5SGA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:28.774412499 (552h4m3.545053834s) [noise=true , address=45.91.116.13:4153 , piece_id=V2I2K74WVNEYDVCJQZMMMVY4SX6CXKSMDQ473ZGHWKC3OAEG5JDQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:42.485153717 (552h4m17.255795052s) [noise=true , address=77.90.31.15:4697 , piece_id=CLSBZ7KD5FLCENIJBC3ME7XXBGXDLU6DOCNT6AAYHO6HLPHVDLYQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:54.799737178 (552h4m29.570378513s) [noise=true , address=77.90.31.13:4626 , piece_id=UP7GSEGMCLWR46BYQEGLYZAW7N4F43ZDOKLN6KJAKAH4OMN6AKEA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: <nil> Jul 16 09:14:55.680653470 (552h4m30.451294805s) [noise=true , address=77.90.31.17:4772 , piece_id=LRVXWHVXOFDU6BRZMEFVNLMJPXWKQD3G5LE35BA5UYEH44SRBPZQ, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57790->77.90.31.17:4772: use of closed network connection Jul 16 09:14:55.680714697 (552h4m30.451356032s) [noise=true , address=77.90.31.18:4806 , piece_id=F5XFYNXII3AL6O74Y52JRDIBV44KIC4TBDJRACKDXDORRQQWCJ6A, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60932->77.90.31.18:4806: use of closed network connection Jul 16 09:14:55.680756227 (552h4m30.451397562s) [noise=true , address=77.90.31.21:4878 , piece_id=W22IHNRS57EZ5KMPR5LP5ELE4KDXQUMOYA3W2GNMCIFDIRDS64TA, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58112->77.90.31.21:4878: use of closed network connection Jul 16 09:14:55.680828745 (552h4m30.45147008s) [noise=true , address=77.90.31.10:4518 , piece_id=WTVEFQBQKDZP7NF4DOIKBWKMXIZSBGKUN57IDIPZ5EHDKE2T7E2Q, seg_pos={0 0}, upload=441503]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42046->77.90.31.10:4518: use of closed network connection Jul 16 09:14:55.680871197 (552h4m30.451512532s) [seg_pos={0 0}, upload=441503]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 09:14:55.927916416 (552h4m30.698557751s) [upload=441503]: Done uploading segments... Jul 16 09:14:55.927940001 (552h4m30.698581336s) [upload=441503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration:* 5.721280619s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)