Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 509821 reports (Page 5511 of 10197)

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.X......................................X.........X.........] duration: 3.445429284s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:22:09 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.445461896s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..X......................................X.........X........] duration: 3.436018577s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:21:00 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.436053032s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.923534609s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:20:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.923564646s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...X......................................X.........X.......] duration: 3.427344628s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:20:17 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.427371239s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.807750231s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:20:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.807784286s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [....X......................................X.........X......] duration: 4.371277102s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:19:43 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.371319484s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.352777633s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:19:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.352814504s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.....X......................................X.........X.....] duration: 3.478530437s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:18:48 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.478560103s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [......X......................................X.........X....] duration: 3.502005728s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:18:19 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.502117421s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.171242354s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:18:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.171274035s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.890242142s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:17:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.890273843s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.......X......................................X.........X...] duration: 3.428701061s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:17:49 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.428728364s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [........X......................................X.........X..] duration: 3.439381191s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:17:27 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.439409146s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.99038525s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:16:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.990410719s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........X......................................X.........X.] duration: 3.50588247s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:16:44 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.50657047s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.264011827s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:15:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.264029541s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..........X......................................X.........X] duration: 5.001237196s trace: https://jaeger.dev.storj.io/trace/7268fb85428c9e52 FAIL (2025-11-06 14:15:47 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001263267s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.301263466s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:15:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.301285598s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.427258021s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:15:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.427289861s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........X......................................X.........] duration: 3.442932329s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:14:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.442967204s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.162782425s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:14:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.1628115s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.319237266s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:13:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.319277724s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............X......................................X........] duration: 3.473216142s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:13:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.473248231s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.36264581s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:13:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.362680225s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............X......................................X.......] duration: 3.660310996s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:13:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.660344248s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............X......................................X......] duration: 4.151496677s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:12:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.151530099s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.829470906s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:12:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.829501284s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.05838453s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:11:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.058422412s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............X......................................X.....] duration: 3.443549908s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:11:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.443578492s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................X......................................X....] duration: 3.46611017s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:10:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.466139184s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.986842151s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:10:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.986872038s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.786572017s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:10:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.786601975s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................X......................................X...] duration: 3.470370234s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:09:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.470397696s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.086462117s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:09:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 6.086493698s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..................X......................................X..] duration: 3.636463794s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:09:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.636495865s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.973865902s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:09:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.973897292s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.816095977s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:08:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.816125364s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...................X......................................X.] duration: 3.499659502s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:08:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.49968486s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.92946921s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:07:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.929501681s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 106.523522ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/419a0cb620bb2df4 FAIL (2025-11-06 14:07:33 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 106.553449ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....................X......................................X] duration: 5.00156685s trace: https://jaeger.dev.storj.io/trace/57f49e58cb7314cb FAIL (2025-11-06 14:07:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001596906s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 76.952694ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/419a0cb620bb2df4 FAIL (2025-11-06 14:07:01 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 76.988631ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 3.440057175s trace: https://jaeger.dev.storj.io/trace/3b18505fce747391 FAIL (2025-11-06 14:06:54 UTC) orbiter warning: context deadline exceeded Duration: 3.440093574s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 348.976852ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3e650e478f46a7f1 FAIL (2025-11-06 14:06:47 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 349.011318ms core website
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.939141271s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:06:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.939172392s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 59.660144ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/419a0cb620bb2df4 FAIL (2025-11-06 14:06:16 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:60910->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 59.687047ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 3.396601906s trace: https://jaeger.dev.storj.io/trace/3b18505fce747391 FAIL (2025-11-06 14:06:02 UTC) orbiter warning: context deadline exceeded Duration: 3.396646991s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.751086925s trace: https://jaeger.dev.storj.io/trace/6f7930158ec18dea FAIL (2025-11-06 14:05:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== dress=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967491189 (170h16m41.663339932s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967873697 (170h16m41.66372245s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.967951827 (170h16m41.66380057s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968261997 (170h16m41.66411075s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968432701 (170h16m41.664281454s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.968895694 (170h16m41.664744447s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969397699 (170h16m41.665246452s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.969914393 (170h16m41.665763146s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.970231888 (170h16m41.666080641s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.971781058 (170h16m41.667629811s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:09.972571684 (170h16m41.668420427s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Uploading piece... Nov 6 13:57:10.827880794 (170h16m42.523729547s) [upload=150053]: Next returned nil segment Nov 6 13:57:10.827905561 (170h16m42.523754314s) [upload=150053]: Waiting for error group managing segments... Nov 6 13:57:11.231640032 (170h16m42.927488785s) [noise=true , address=81.171.14.70:4042 , piece_id=YCONCKCIBPX7KMMAR5MEDBIY5CR7ZW26AWUHLB4EWRFMEDQYLD7Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.256442090 (170h16m42.952290843s) [noise=true , address=81.171.14.70:4043 , piece_id=SB4C6URXGDMVMMEL4KEKILGUEWQKQL5OOQOTPIVNM64S4EZTJJWQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.257443336 (170h16m42.953292089s) [noise=true , address=81.171.14.70:4041 , piece_id=RDIDN2YMS4FZYBKQPPQYZKEEPDMVGSTKCTHGPIMKKM7IPFV5YDBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258387294 (170h16m42.954236047s) [noise=true , address=89.149.201.147:4015 , piece_id=QIRYWFPQR5NTPI4O4MZAFBFSDJ64X6QNGFSXGUIGXES5MLPYKWBQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.258894860 (170h16m42.954743603s) [noise=true , address=89.149.201.147:4005 , piece_id=NUBPG2CZKYOFKKEDATQNMIIEMRJLKCV7BXON74JJTKOZ4O3RLXYA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.261496525 (170h16m42.957345278s) [noise=true , address=89.149.201.147:4020 , piece_id=JVF7STR6OLUCTUSYV6CXFVIKVATTOJHEAEJIQ4GQ6PYBSY5JIWHA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.272583977 (170h16m42.968432741s) [noise=true , address=89.149.201.147:4017 , piece_id=NM7LFEV6SHZVSEVMYISA6HXJ35MNPNHXW73OLALOUHZHNSFUD25Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.278466539 (170h16m42.974315282s) [noise=true , address=81.171.14.70:4034 , piece_id=UY72SXBSXJEDYGSBDCR2EQQU7ONF4JECXUYCHNFFVLEBPB7EP3SA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.306805412 (170h16m43.002654165s) [noise=true , address=89.149.201.147:4009 , piece_id=DMSFQWMG6LGGW5RIXHNAFSMTB4NQOLGRDOCQNM2P7HQ7PB7XIOAQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.307888204 (170h16m43.003736967s) [noise=true , address=89.149.201.147:4010 , piece_id=AXUNULAVIM67QI4RV52PJCWXGFHS2WFDBX4R57MSRRTC4NPN5JLQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.343739372 (170h16m43.039588125s) [noise=true , address=89.149.201.147:4012 , piece_id=KLXS663OUTFUL2FPSTCJPP5N5BGDC7O5JCKSMEP4S633GNB6HAJA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.716425841 (170h16m43.412274594s) [noise=true , address=89.149.201.147:4024 , piece_id=BRGY7OUUISINCEPND2NESFP3XZ7FLG4DFW3RNDYXJUHXSKGGZ4DQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.753901162 (170h16m43.449749925s) [noise=true , address=89.149.201.147:4023 , piece_id=4JE3VN4QUOJOJQ36ELHXGSTYM6RVSAWJVGVCFMEYVYU22YZ2YXTQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.805773276 (170h16m43.501622029s) [noise=true , address=89.149.201.147:4011 , piece_id=KFKJYMKWZSIYQ46NMLBKY24BYPDRWCRG7TAJEIW55JIHR6T5R3KA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:11.815857448 (170h16m43.511706191s) [noise=true , address=89.149.201.147:4013 , piece_id=HE7XBI3JR45JCMXGUPGCIWEJANBU7BIFF3JVDARYIZEZUBDMVXXA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.467950153 (170h16m44.163798906s) [noise=true , address=185.167.204.205:4013 , piece_id=KBYH7LUODR6RSEKOCWZN46PATU5OSKSQF2AUTN45KZYZOXYRZC3Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.481087159 (170h16m44.176935912s) [noise=true , address=185.167.204.205:4021 , piece_id=OLV3IKKXTCXKOUSZF2NTTDQXA46SF4TATXICAEWFJQGSMKLNB2YQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.485759787 (170h16m44.18160854s) [noise=true , address=185.167.204.206:4056 , piece_id=SUP37NDMXS4CDS5Z7XSPABOTYHWCGVVJCPRTSTXZ4S4KQN73I7HQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.491917152 (170h16m44.187765905s) [noise=true , address=185.167.204.205:4014 , piece_id=D7VDIU7AKGGT2JVZXBSMXM2DKG2ILJND5SGE2EHHJDKL7QVZQJZA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.496242337 (170h16m44.19209109s) [noise=true , address=185.167.204.205:4010 , piece_id=VQN66GUL5K3MTZKK4ZD2ZEF47BFXFC6KTGYNL3W4YYY3BANPZRJQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.503381952 (170h16m44.199230705s) [noise=true , address=185.167.204.205:4035 , piece_id=PHCM37Y4ERDT3LXI2OIJCXDY6TC4ZTAHEOJKJNERQD7XY2I2VLCQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.508231517 (170h16m44.204080281s) [noise=true , address=185.167.204.206:4058 , piece_id=4VHBMCJBKLMPI2ARQFFPS5CSSMY6RYVB24ERFARVGDWR3NTJR4IA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.552927117 (170h16m44.24877587s) [noise=true , address=185.167.204.205:4007 , piece_id=J5ZPRWBIY3ATFERVVCEBLRYI6MXEURNZA6KAVCVX4Y5FEGY6H4IQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.554453173 (170h16m44.250301926s) [noise=true , address=185.167.204.206:4040 , piece_id=UNZQTD5MKIWKVAH7BMCIQD527PIWTGOYEYXTVYIZVVW2PVI7XUKQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:57:12.555070939 (170h16m44.250919692s) [noise=true , address=185.167.204.205:4034 , piece_id=M3AEVYF5GSIJIBHDF5W22PEILMREFIP76J6MQ6L25VZQJWFKEGXQ, seg_pos={0 0}, upload=150053]: Done uploading piece. err: <nil> Nov 6 13:58:09.307307484 (170h17m41.003156237s) [noise=true , address=185.167.204.205:4026 , piece_id=LUUWBTTDXT46HJOYAXPUQVOASM7E34JFB5ATDZIXQ7VQSVFNUSCA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:34426->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307473691 (170h17m41.003322464s) [noise=true , address=185.167.204.206:4072 , piece_id=X62JXCQKFGXZNUFIAJ6TQT2NQO5CEJFRKE5TRJ7XCUZVMSB2YDBA, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 188.166.225.170:34834->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307485523 (170h17m41.003334266s) [noise=true , address=185.167.204.206:4066 , piece_id=ZKA5JBI26A332CUNON7FMGEERZWWZ2A22SRAV254HE2B23F25K2Q, seg_pos={0 0}, upload=150053]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: write tcp 188.166.225.170:38838->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 6 13:58:09.307508226 (170h17m41.003356979s) [seg_pos={0 0}, upload=150053]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 6 13:58:09.482426726 (170h17m41.178275479s) [upload=150053]: Done uploading segments... Nov 6 13:58:09.482474207 (170h17m41.17832296s) [upload=150053]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.751118005s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 630.565806ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3e650e478f46a7f1 FAIL (2025-11-06 14:05:46 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42814->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 630.600222ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 4.427683629s trace: https://jaeger.dev.storj.io/trace/3b18505fce747391 FAIL (2025-11-06 14:05:26 UTC) orbiter warning: context deadline exceeded Duration: 4.42772601s core uplink us-select-1 us1-placement-20

1
...
5509
5510
5511
5512
5513
...
10197