Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.851730525s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:05:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.851758276s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [X..X...........] duration:
76.93984ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 14:05:15 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 76.969044ms
core
website
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.419746799s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:04:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 6.419780333s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [..X............] duration:
332.61709ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 14:04:48 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 332.64301ms
core
website
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
6.315485998s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:04:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 6.315517136s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [.X..X..........] duration:
149.208753ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 14:04:29 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 149.230015ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [X.......................X...................................] duration:
3.400134885s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 14:04:26 UTC)
orbiter warning: context deadline exceeded
Duration: 3.400165985s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.845611649s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:04:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.845642087s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [..X..X.........] duration:
78.784943ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 14:03:57 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 78.807405ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [...X...........] duration:
358.967989ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 14:03:57 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 358.985131ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [....X..........] duration:
358.208281ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 14:03:40 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 358.238108ms
core
website
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.950327291s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:03:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.950361087s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.X.......................X..................................] duration:
3.407274437s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 14:03:15 UTC)
orbiter warning: context deadline exceeded
Duration: 3.40734464s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [...X..X........] duration:
75.544923ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 14:03:10 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 75.578758ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.....X.........] duration:
353.902932ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 14:03:01 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 353.933339ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..X.......................X.................................] duration:
3.420900748s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 14:02:52 UTC)
orbiter warning: context deadline exceeded
Duration: 3.421079698s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.103427324s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:02:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 6.103456489s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...X.......................X................................] duration:
3.419529758s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 14:02:30 UTC)
orbiter warning: context deadline exceeded
Duration: 3.419551621s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [....X..X.......] duration:
78.970546ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 14:02:16 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 78.993579ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [....X.......................X...............................] duration:
3.508399072s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 14:02:06 UTC)
orbiter warning: context deadline exceeded
Duration: 3.508430973s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [......X........] duration:
344.444501ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 14:01:55 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 344.473146ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [.....X..X......] duration:
101.029652ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 14:01:43 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 101.058345ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.....X.......................X..............................] duration:
3.436336685s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 14:01:40 UTC)
orbiter warning: context deadline exceeded
Duration: 3.436870631s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
6.060082957s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:01:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 6.060294602s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.......X.......] duration:
393.868525ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 14:01:05 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 393.982513ms
core
website
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.772079066s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:01:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 5.772106339s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [......X.......................X.............................] duration:
3.670828401s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 14:01:01 UTC)
orbiter warning: context deadline exceeded
Duration: 3.670871783s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [......X..X.....] duration:
62.443986ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 14:00:48 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 62.476068ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [........X......] duration:
346.735225ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 14:00:39 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 346.765945ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.......X.......................X............................] duration:
3.528929424s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 14:00:26 UTC)
orbiter warning: context deadline exceeded
Duration: 3.528960583s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [.......X..X....] duration:
117.233566ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 14:00:22 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 117.264135ms
core
website
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
6.243686329s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 14:00:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 6.243718961s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.........X.....] duration:
318.769884ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 14:00:15 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 318.792706ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [........X.......................X...........................] duration:
3.419602365s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 13:59:55 UTC)
orbiter warning: context deadline exceeded
Duration: 3.41963075s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [........X..X...] duration:
91.241071ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 13:59:20 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 91.270035ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........X.......................X..........................] duration:
3.426015068s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 13:59:17 UTC)
orbiter warning: context deadline exceeded
Duration: 3.426044535s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
6.250732107s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 13:59:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 6.250777883s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [..........X....] duration:
322.726025ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 13:59:05 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 322.767956ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..........X.......................X.........................] duration:
3.408120132s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 13:58:35 UTC)
orbiter warning: context deadline exceeded
Duration: 3.408158246s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [.........X..X..] duration:
92.627521ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 13:58:29 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 92.651367ms
core
website
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
6.187665301s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 13:58:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 6.187687783s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [...........X...] duration:
649.49601ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 13:58:12 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 649.52786ms
core
website
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.179439226s trace:
https://jaeger.dev.storj.io/trace/6f7930158ec18dea
FAIL (2025-11-06 13:58:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece...
Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment
Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments...
Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil>
Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments...
Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 1m0.179471577s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...........X.......................X........................] duration:
3.510799141s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 13:58:02 UTC)
orbiter warning: context deadline exceeded
Duration: 3.510834629s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
2 out of 15 is failed (threshold=6) [..........X..X.] duration:
154.892287ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 13:57:41 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 154.914479ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [............X.......................X.......................] duration:
3.475551103s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 13:57:23 UTC)
orbiter warning: context deadline exceeded
Duration: 3.475573777s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [............X..] duration:
320.200616ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 13:57:12 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 320.248517ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)antiblip:
3 out of 15 is failed (threshold=6) [X..........X..X] duration:
1.477438535s remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/ trace:
https://jaeger.dev.storj.io/trace/419a0cb620bb2df4
FAIL (2025-11-06 13:57:00 UTC)
orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 1.47753591s
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.............X.] duration:
346.480789ms remote:
"tcp:[2620:cb:2000::1]:443" target:
https://www.storj.io/dcs-satellites trace:
https://jaeger.dev.storj.io/trace/3e650e478f46a7f1
FAIL (2025-11-06 13:56:46 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer
Duration: 346.512628ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.............X.......................X......................] duration:
3.456448487s trace:
https://jaeger.dev.storj.io/trace/3b18505fce747391
FAIL (2025-11-06 13:56:39 UTC)
orbiter warning: context deadline exceeded
Duration: 3.456523309s
core
uplink
us-select-1
us1-placement-20
|