Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
713.50857ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:25:56 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 713.584224ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
746.971117ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:25:37 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 747.004061ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.754706077s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:25:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.754734469s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
702.22338ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:24:24 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 702.251744ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.703877548s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:24:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.703908376s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [X..........X................................................] duration:
5.82219672s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:23:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.822228421s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
698.533483ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:23:31 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 698.561477ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
708.704341ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:23:00 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 708.733236ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.X..........X...............................................] duration:
6.916921514s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:22:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 6.916950108s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..X..........X..............................................] duration:
5.899211082s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:22:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.899247411s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
718.051799ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:22:28 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 718.078661ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...X..........X.............................................] duration:
5.698434405s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:21:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.698512674s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
709.901841ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:21:36 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 709.934382ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....X..........X............................................] duration:
10.390057661s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:20:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 10.390096725s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
763.858674ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:20:36 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 763.926263ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....X..........X...........................................] duration:
5.821782632s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:20:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.822370082s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......X..........X..........................................] duration:
5.568104402s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:19:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.568138808s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
748.429855ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:19:28 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 748.461595ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......X..........X.........................................] duration:
5.799439355s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:19:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.799492326s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [X.......X..........X........................................] duration:
5.900544039s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:18:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.900575308s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
754.063242ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:18:13 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 754.095252ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
759.2195ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:17:54 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 762.403514ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.X.......X..........X.......................................] duration:
5.655968464s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:17:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.655999595s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
751.972471ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:17:29 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 752.035632ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [..X.......X..........X......................................] duration:
5.761863817s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:17:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.761896799s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [...X.......X..........X.....................................] duration:
5.853640405s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:16:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.853672496s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
758.756168ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:16:29 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 758.785825ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [....X.......X..........X....................................] duration:
5.790437565s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:15:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.790469817s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
736.719368ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:15:28 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 736.754122ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.....X.......X..........X...................................] duration:
5.833488712s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:14:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.833600285s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
743.20229ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:14:25 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 743.22809ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
4 out of 60 is failed (threshold=3) [X.....X.......X..........X..................................] duration:
5.780926815s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:13:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.780957604s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
737.321012ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:13:37 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 737.363953ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
4 out of 60 is failed (threshold=3) [.X.....X.......X..........X.................................] duration:
5.765974065s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:12:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.76600781s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
741.865506ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:12:48 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 741.897336ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
707.096019ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:12:04 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 707.685332ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
4 out of 60 is failed (threshold=3) [..X.....X.......X..........X................................] duration:
5.62960967s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:11:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.629639777s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
761.925035ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:11:17 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 761.95946ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [X..X.....X.......X..........X...............................] duration:
5.703447592s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:10:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.703480466s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
698.625779ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:10:39 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 698.678159ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
758.09017ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:10:06 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 758.112351ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [.X..X.....X.......X..........X..............................] duration:
5.70236453s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:09:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.702394737s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [..X..X.....X.......X..........X.............................] duration:
5.678009231s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:09:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.678036804s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
712.552107ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:09:22 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 712.568669ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
732.417093ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:08:53 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 732.44665ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
720.25168ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:08:21 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 720.291335ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [...X..X.....X.......X..........X............................] duration:
5.751056515s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:08:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.751089539s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [....X..X.....X.......X..........X...........................] duration:
5.845600174s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:07:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.845633066s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
715.723446ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/30eab70780772bd4
FAIL (2025-11-18 18:07:24 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": read tcp 188.166.225.170:40922->45.38.24.2:443: read: connection reset by peer
Duration: 715.760087ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [.....X..X.....X.......X..........X..........................] duration:
5.883794886s trace:
https://jaeger.dev.storj.io/trace/1672808e7b18f6b7
FAIL (2025-11-18 18:07:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
425857s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648487689 (225h44m57.19995815s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.648948867 (225h44m57.200419328s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653049836 (225h44m57.204520307s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.653709502 (225h44m57.205179963s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.654369940 (225h44m57.205840401s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655164553 (225h44m57.206635014s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.655752162 (225h44m57.207222623s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.656381360 (225h44m57.207851811s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:58.657015407 (225h44m57.208485868s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Uploading piece...
Nov 18 17:48:59.925639703 (225h44m58.477110164s) [upload=198823]: Next returned nil segment
Nov 18 17:48:59.925658439 (225h44m58.4771289s) [upload=198823]: Waiting for error group managing segments...
Nov 18 17:49:00.516140593 (225h44m59.067611054s) [noise=true , address=185.167.204.206:4048 , piece_id=TBETKXWWIAP7QTLHM2MR6O2ELAR5KXJVDZJ6YHA2SZWHHBA43QZQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.525951254 (225h44m59.077421715s) [noise=true , address=185.167.204.205:4024 , piece_id=YOKYH266I5HDH7ODITMSRQH6BRF6SLSYSYTPLJJV2YU2A3UWU4QA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:00.572723186 (225h44m59.124193647s) [noise=true , address=185.167.204.205:4009 , piece_id=PFT2TC5KLIS6KOQO2BYLNO3LTQRZZERXSZQMYOEBEPS3JTFLWKNQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.067060302 (225h44m59.618530763s) [noise=true , address=89.149.201.147:4001 , piece_id=VL52RDB5AZ3RDN43D6ZT57FPWZBSVYJ2T56MPJ6HW4MCQ7SML4SA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.075794143 (225h44m59.627264604s) [noise=true , address=185.167.204.206:4049 , piece_id=IGLUPJ3L5V3TIULP3PQCD2GEEC3B6BWEGUNJ5S7E7KAPALHDRVTQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081180189 (225h44m59.63265065s) [noise=true , address=185.167.204.205:4032 , piece_id=UOFV5UCAVUYM4UDBBGOHXM66KENDHTXG55CJAA7KUQONETTFS4GQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.081984520 (225h44m59.633454981s) [noise=true , address=89.149.201.147:4005 , piece_id=MODXJYLKY4KAIMISPC3SUMRT5EVSP7TMCG4DYSDQII2IHOSHH4TQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.105696722 (225h44m59.657167193s) [noise=true , address=89.149.201.147:4022 , piece_id=ZHX2SXDJ5TKERC3PZ3OYFKNFUDQI77EDLCXEPHMG7KPSWGCW7BUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.191257906 (225h44m59.742728367s) [noise=true , address=89.149.201.147:4019 , piece_id=XKQ54YM73TJPK7S3ODN7F2DMABIBV7DXEBLPZUAWQUR5HTS27T3A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.202193348 (225h44m59.75366381s) [noise=true , address=89.149.201.147:4003 , piece_id=SNGTWOIOJ43WD46C47MDL5AZ23P5V6Z4REGMMHX2MJ3AQZQYHBSA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.272273634 (225h44m59.823744095s) [noise=true , address=185.167.204.206:4061 , piece_id=SGKQYVH2EEOP7EZKBILSN6TAXIOMSBCHXANJRTG6UHFJGZMG2YJA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.305099009 (225h44m59.856569471s) [noise=true , address=185.167.204.205:4035 , piece_id=UFWY2PFFJEAIQEH3HFOA6AAJAJB5XWZHKRLA77P2OES4U3GX5CXA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.821876776 (225h45m0.373347247s) [noise=true , address=185.167.204.205:4021 , piece_id=QOIYSH6CGSUVKSYCPEBTCKPV6K272ABLCXCAVWHTIFWR7OK2BC5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.921180269 (225h45m0.47265073s) [noise=true , address=185.167.204.205:4034 , piece_id=CWYABDK27G7HBIE4JBHNUGSAWRQ4LVM6COXI65XHIYXF4UENCU7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:01.926801301 (225h45m0.478271773s) [noise=true , address=185.167.204.205:4028 , piece_id=IQVISGBQMRJTJCQ65X6CVKFQWPZOXWX5QDWXSAFAWRXPM5WKFTCQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.344156560 (225h45m0.895627021s) [noise=true , address=185.167.204.206:4069 , piece_id=4KNW6MQ6IXIWQOWU7FTEYY5EZY757YNWY3DSSZMGCP3EQAAFVF6A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.349980540 (225h45m0.901451021s) [noise=true , address=185.167.204.205:4015 , piece_id=4BTQP2E7XC2IZPDVVQIQ2F7OZT2LDFASRSATIM3CG6ZTXRGEOKUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:02.888522040 (225h45m1.439992491s) [noise=true , address=185.167.204.205:4002 , piece_id=I7UOJPUNYUTCCKRIQVWQLV7T3QESQEGU2N2F2Q6BBZC642P4WEUQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.342026652 (225h45m1.893497113s) [noise=true , address=185.167.204.206:4046 , piece_id=WNGNA6UGVEUAVMJVRKBLWEATJOX6OKCE2KDUNQ3VUXAVIQ6FP77Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:03.582256503 (225h45m2.133726964s) [noise=true , address=185.167.204.206:4047 , piece_id=VAIEZUOZPZ2DF5FJ2RRFDFMY4IVFUQBZAV7AJA3EZYCXMLVKNMFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.653771287 (225h45m40.205241738s) [noise=true , address=81.171.14.70:4031 , piece_id=24GPZJLCMFY57NBCE3Z3JN7FW73AOQVLZMGHV55NTBSK7JDMIL5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:41.728831976 (225h45m40.280302427s) [noise=true , address=81.171.14.70:4027 , piece_id=XF53EHZZHERKAIRLZYNDXGPMZAQBZKRZZFXIHZMPF6FJEQ6FVUWA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:46.009388378 (225h45m44.560858839s) [noise=true , address=81.171.14.70:4035 , piece_id=BZSYW22GJOT774WJQ3QFJIINYQX5LWMJ4XC63AXPFZBRGG2FCZ5A, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:50.272884604 (225h45m48.824355055s) [noise=true , address=81.171.14.70:4025 , piece_id=GKLEGPL5YPRFVQOF3NQ6I7MHGQ6YYM34NNMZCLY46X3BS4YH6G7Q, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:51.451202409 (225h45m50.00267287s) [noise=true , address=81.171.14.70:4037 , piece_id=FU5WM2VPV6U5J65RELXUJZ6IZ2H2DWZUAKWEIAMXKPCIMV5UTYOQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: <nil>
Nov 18 17:49:57.987545771 (225h45m56.539016233s) [noise=true , address=81.171.14.70:4038 , piece_id=LVDTLLPTEU3QFVWOL74Y4LCOBLZPHWYO72KES2IDFGZDOLVWJZAQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35454->81.171.14.70:4038: use of closed network connection
Nov 18 17:49:57.988053327 (225h45m56.539523788s) [noise=true , address=81.171.14.70:4045 , piece_id=75QBSUKTJZMS4G7PGRGGICX3IPFVCAH6DQO7S5O5LMQEXEBRTIFA, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49424->81.171.14.70:4045: use of closed network connection
Nov 18 17:49:57.988179428 (225h45m56.539649879s) [noise=true , address=81.171.14.70:4039 , piece_id=UZDUYKNGEJQGFLAKFHC4OSPIOMZ4CC25DWA5U363SN5QO2DSKVVQ, seg_pos={0 0}, upload=198823]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->81.171.14.70:4039: use of closed network connection
Nov 18 17:49:57.988220887 (225h45m56.539691348s) [seg_pos={0 0}, upload=198823]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 18 17:49:58.172771756 (225h45m56.724242217s) [upload=198823]: Done uploading segments...
Nov 18 17:49:58.172825708 (225h45m56.724296169s) [upload=198823]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.883827747s
core
uplink
eu-select-1
eu1-placement-15
|