Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 427923 reports (Page 8210 of 8559)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...................................................XXXX.XXXX] duration: 30.23210605s FAIL (2025-07-11 22:07:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== tarting upload Jul 11 22:07:04.500584501 (444h56m39.271225846s) [upload=355868]: Done starting upload Jul 11 22:07:04.500760858 (444h56m39.271402193s) [upload=355868]: Uploading segments... Jul 11 22:07:04.500789072 (444h56m39.271430407s) [upload=355868]: Got next segment. Inline: false Jul 11 22:07:04.500811586 (444h56m39.271452921s) [upload=355868]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:07:04.500816254 (444h56m39.271457579s) [upload=355868]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:07:04.735319972 (444h56m39.505961307s) [seg_pos={0 0}, upload=355868]: Begin upload segment... Jul 11 22:07:04.735379345 (444h56m39.5060207s) [seg_pos={0 0}, upload=355868]: Done begin upload segment. Jul 11 22:07:04.735415885 (444h56m39.50605724s) [noise=true , address=104.36.82.11:4113 , piece_id=PJ76UVQTN4YOH5WK6Z7AYZOOXA7LNITM3Y2GKXFNRO6IMDM4OPMQ, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.735471983 (444h56m39.506113328s) [noise=true , address=104.36.82.6:4875 , piece_id=3XFZRTUJSHULLMX65ATIU5Z2GEV26OSG6WCKOMNNECXIQJ6E2TWQ, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.736096467 (444h56m39.506737792s) [noise=true , address=77.90.31.10:4517 , piece_id=G6YSAI2EV4RBDBVFKQ7KUKRULMBE5VCVVTSAMRBXKUIW6IBP2BBA, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.736197321 (444h56m39.506838656s) [noise=true , address=104.36.82.14:4222 , piece_id=BX3NL26QTFUYNNF4HFPZOTZIJO53QD54RYBVCLN2FZHLMR4DA2FQ, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.736641931 (444h56m39.507283266s) [noise=true , address=45.91.116.8:4047 , piece_id=SL52JTSO2TXFOCGCSFEA77BFUXHUYCUSGLT4XUDS6NF2DZ5L2RVA, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.737042026 (444h56m39.507683371s) [noise=true , address=104.36.82.8:4970 , piece_id=YC4STMKVSIMZO2NXATUWNNL4HWTXKH75MOWETFKKNMBW5BKKQKBQ, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.737392717 (444h56m39.508034052s) [noise=true , address=77.90.31.13:4627 , piece_id=URBCLNAFVEE7OYZVL6KOMQUNDHKOQ67WK4QPVTILA2AHXQMXGP3A, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.737758116 (444h56m39.508399451s) [noise=true , address=77.90.31.11:4554 , piece_id=4RQMQ44PE6LD2YLXE5ZMAVY2P2QYYAL42A6QPTCKZLXPUQXGDJVA, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.738242102 (444h56m39.508883437s) [noise=true , address=104.36.82.16:4295 , piece_id=BXIOPOVENMAHFXCZRGOMB4AF2TU2MWBHN7NWFY4GKNVQTMYED7AA, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.738562305 (444h56m39.50920364s) [noise=true , address=104.36.82.15:4257 , piece_id=PZY4AFYUD3AUK6A2VF4A3EJGCLVH5ZYQF3WTZAMHP2MX7PIGACFQ, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.738869503 (444h56m39.509510838s) [noise=true , address=104.36.82.3:4767 , piece_id=WIWAEGX6P7VM3LNUJJ6RET72DPTQGQRTNCXU3FI33JAMM4HHIYUA, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.739164677 (444h56m39.509806012s) [noise=true , address=45.91.116.13:4155 , piece_id=6E23AOEFD54YIMFCVLUN3QGSEOI44R5SHOMRTXG3IJ7IWBEWLYVQ, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.739461766 (444h56m39.510103101s) [noise=true , address=45.91.116.5:4299 , piece_id=HPQYGGNIHNEPPJGYTEIXTNNYVXB32X5S2XP4KVHZ5SZEHWWM22EA, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.739705382 (444h56m39.510346717s) [noise=true , address=77.90.31.18:4805 , piece_id=WO3RZKKF5E2OT36VZQCOUOB3TB742OCJIEMMY4BQWNHWHVG4HZQA, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.739942817 (444h56m39.510584142s) [noise=true , address=104.36.82.5:4840 , piece_id=AQHIEOCMC2KJD3VTKFDYCYEVTFATX27L3ZZBSGGX22OOLYCPTP6A, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:04.740193506 (444h56m39.510834841s) [noise=true , address=77.90.31.12:4592 , piece_id=USIXD7X43GMFI4IOJKQKRKXFWCNG3CZEIU4JWELRZ7ZC2YHJJ4QA, seg_pos={0 0}, upload=355868]: Uploading piece... Jul 11 22:07:06.494904742 (444h56m41.265546077s) [upload=355868]: Next returned nil segment Jul 11 22:07:06.494924751 (444h56m41.265566086s) [upload=355868]: Waiting for error group managing segments... Jul 11 22:07:06.794216407 (444h56m41.564857742s) [noise=true , address=45.91.116.8:4047 , piece_id=SL52JTSO2TXFOCGCSFEA77BFUXHUYCUSGLT4XUDS6NF2DZ5L2RVA, seg_pos={0 0}, upload=355868]: Done uploading piece. err: <nil> Jul 11 22:07:06.823773854 (444h56m41.594415179s) [noise=true , address=45.91.116.13:4155 , piece_id=6E23AOEFD54YIMFCVLUN3QGSEOI44R5SHOMRTXG3IJ7IWBEWLYVQ, seg_pos={0 0}, upload=355868]: Done uploading piece. err: <nil> Jul 11 22:07:07.220693587 (444h56m41.991334922s) [noise=true , address=45.91.116.5:4299 , piece_id=HPQYGGNIHNEPPJGYTEIXTNNYVXB32X5S2XP4KVHZ5SZEHWWM22EA, seg_pos={0 0}, upload=355868]: Done uploading piece. err: <nil> Jul 11 22:07:09.058442054 (444h56m43.829083389s) [noise=true , address=77.90.31.13:4627 , piece_id=URBCLNAFVEE7OYZVL6KOMQUNDHKOQ67WK4QPVTILA2AHXQMXGP3A, seg_pos={0 0}, upload=355868]: Done uploading piece. err: <nil> Jul 11 22:07:10.334832075 (444h56m45.10547341s) [noise=true , address=77.90.31.10:4517 , piece_id=G6YSAI2EV4RBDBVFKQ7KUKRULMBE5VCVVTSAMRBXKUIW6IBP2BBA, seg_pos={0 0}, upload=355868]: Done uploading piece. err: <nil> Jul 11 22:07:10.350870038 (444h56m45.121511363s) [noise=true , address=77.90.31.12:4592 , piece_id=USIXD7X43GMFI4IOJKQKRKXFWCNG3CZEIU4JWELRZ7ZC2YHJJ4QA, seg_pos={0 0}, upload=355868]: Done uploading piece. err: <nil> Jul 11 22:07:12.016007617 (444h56m46.786648952s) [noise=true , address=77.90.31.11:4554 , piece_id=4RQMQ44PE6LD2YLXE5ZMAVY2P2QYYAL42A6QPTCKZLXPUQXGDJVA, seg_pos={0 0}, upload=355868]: Done uploading piece. err: <nil> Jul 11 22:07:20.168980913 (444h56m54.939622278s) [noise=true , address=77.90.31.18:4805 , piece_id=WO3RZKKF5E2OT36VZQCOUOB3TB742OCJIEMMY4BQWNHWHVG4HZQA, seg_pos={0 0}, upload=355868]: Done uploading piece. err: <nil> Jul 11 22:07:33.844664383 (444h57m8.615305718s) [noise=true , address=104.36.82.6:4875 , piece_id=3XFZRTUJSHULLMX65ATIU5Z2GEV26OSG6WCKOMNNECXIQJ6E2TWQ, seg_pos={0 0}, upload=355868]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: rpc: dial tcp 104.36.82.6:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:33.844680132 (444h57m8.615321457s) [noise=true , address=104.36.82.3:4767 , piece_id=WIWAEGX6P7VM3LNUJJ6RET72DPTQGQRTNCXU3FI33JAMM4HHIYUA, seg_pos={0 0}, upload=355868]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 188.166.225.170:49576->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:33.844725049 (444h57m8.615366374s) [noise=true , address=104.36.82.16:4295 , piece_id=BXIOPOVENMAHFXCZRGOMB4AF2TU2MWBHN7NWFY4GKNVQTMYED7AA, seg_pos={0 0}, upload=355868]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: rpc: dial tcp 104.36.82.16:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:33.844767080 (444h57m8.615408415s) [noise=true , address=104.36.82.14:4222 , piece_id=BX3NL26QTFUYNNF4HFPZOTZIJO53QD54RYBVCLN2FZHLMR4DA2FQ, seg_pos={0 0}, upload=355868]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:33.844785504 (444h57m8.615426839s) [noise=true , address=104.36.82.5:4840 , piece_id=AQHIEOCMC2KJD3VTKFDYCYEVTFATX27L3ZZBSGGX22OOLYCPTP6A, seg_pos={0 0}, upload=355868]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: write tcp 188.166.225.170:33444->104.36.82.5:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:33.844794522 (444h57m8.615435847s) [noise=true , address=104.36.82.11:4113 , piece_id=PJ76UVQTN4YOH5WK6Z7AYZOOXA7LNITM3Y2GKXFNRO6IMDM4OPMQ, seg_pos={0 0}, upload=355868]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:33.844806174 (444h57m8.615447509s) [noise=true , address=104.36.82.8:4970 , piece_id=YC4STMKVSIMZO2NXATUWNNL4HWTXKH75MOWETFKKNMBW5BKKQKBQ, seg_pos={0 0}, upload=355868]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 188.166.225.170:43668->104.36.82.8:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:07:33.844862181 (444h57m8.615503516s) [noise=true , address=104.36.82.15:4257 , piece_id=PZY4AFYUD3AUK6A2VF4A3EJGCLVH5ZYQF3WTZAMHP2MX7PIGACFQ, seg_pos={0 0}, upload=355868]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: context deadline exceeded; context deadline exceeded Jul 11 22:07:33.844883672 (444h57m8.615525048s) [seg_pos={0 0}, upload=355868]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 11 22:07:34.074840011 (444h57m8.845481346s) [upload=355868]: Done uploading segments... Jul 11 22:07:34.074867513 (444h57m8.845508848s) [upload=355868]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.232129624s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XX.X...] duration: 14.70068022s FAIL (2025-07-11 22:07:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.700694677s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XX.X..] duration: 38.223551596s FAIL (2025-07-11 22:07:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 38.223574459s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...................................................X.XXX.XXX] duration: 30.000518235s FAIL (2025-07-11 22:07:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000539555s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......................................................X..XX.] duration: 35.423597968s FAIL (2025-07-11 22:07:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 47]: Done begin upload segment. Jul 11 22:05:25.250192172 (444h55m0.020833497s) [noise=true , address=104.36.82.3:4799 , piece_id=E3PJM326VZOUQQIXYYK2T3IDEDGTHYR6M2GNPZECR4YXJRSJGCTA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.250231017 (444h55m0.020872352s) [noise=true , address=77.90.31.11:4588 , piece_id=QKDMMYO5KGDLTFB74BSPG7UYP6VWEUQBS52WPLW2VKDQHXM4LNOQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.250569715 (444h55m0.02121105s) [noise=true , address=45.91.95.5:4034 , piece_id=CCLUI4ELT3AMG62RKTLXEGM4LBCQFTQQJHKKAD7LMOFE4XVYTWUA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.250766191 (444h55m0.021407526s) [noise=true , address=77.90.31.12:4619 , piece_id=V2CFGF4J3CRRMQVBVUTYFOAU6NHNW7ZZNXHLW3B5WC6RTW6MSJZQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.250847707 (444h55m0.021489042s) [noise=true , address=45.91.116.11:4149 , piece_id=JJ735AOMNAZ5V2BWJGA2BPLQ375574VANFAFBGXAOVNODKGD4K5Q, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251107715 (444h55m0.02174905s) [noise=true , address=104.36.82.19:4414 , piece_id=4YUT5MJANGYRZUSAVBJ4NKZH5V5XTTMQI4OZX5WBZSMNS7IRJDPA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251178610 (444h55m0.021819945s) [noise=true , address=79.127.220.101:4059 , piece_id=5Y2CK5SM54UODEFCGPDTAAAMF3JRC5CV7AGQYCZ6PW47EQD5VV2Q, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251489936 (444h55m0.022131341s) [noise=true , address=104.36.82.51:4081 , piece_id=YQKYMQJ676ODM3ZTSHRQANURNQB4WKQS7HW34AB4G35FVYYC5FAA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251521937 (444h55m0.022163272s) [noise=true , address=77.90.31.21:4910 , piece_id=R43OAJYZUOW3ZNG2PWF7BMXRO37U44MFEAGPEAPQCZNUP6GUEBIQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251908417 (444h55m0.022549742s) [noise=true , address=77.90.31.13:4640 , piece_id=DUQI7XZUGJFKCW5D73FZGEDSLR3S5PKCPPH6V2XLWSR35BUQZ54A, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.252223779 (444h55m0.022865115s) [noise=true , address=104.36.82.6:4901 , piece_id=SE4Y3VRZRUWDBBIQFYCRDH3KYSPWFSN4KOCS3KKUMLLTDRX7S3PA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.253148930 (444h55m0.023790265s) [noise=true , address=45.91.95.2:4021 , piece_id=37GWHZ27ALNE3XJWYMRZ3QDB6HYL6JL5KSSRMR3BQMBMDEMQ6NAA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.253471086 (444h55m0.024112411s) [noise=true , address=45.91.95.4:4021 , piece_id=NMZAYDJGGRU27TY7QCKYDGGFWBWMWWKHIROPT57QVUSC4HKAUHMQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.253751101 (444h55m0.024392436s) [noise=true , address=104.36.82.50:4035 , piece_id=RPDFNBKOYV4HTINO32MZ5K7ESLNDTUPSAHXKLKJ2J7XWWV7RJGOQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.254046447 (444h55m0.024687802s) [noise=true , address=77.90.31.15:4707 , piece_id=IULW4EB4E6R56FTK3ZT527MYU22CQFW7XMWUC7ULUGJPS5VSL2ZA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.254400644 (444h55m0.025041979s) [noise=true , address=104.36.82.4:4820 , piece_id=7DBP4GL3YMF23V3U27ZS6STMJWS2FLY4HWDKHBQL67A2C3SKUDWA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.254676783 (444h55m0.025318108s) [noise=true , address=79.127.205.233:4656 , piece_id=4YNKONSSBYFTV4CCGI3JKAWSADBEOBKKE24L7EEO64CWUZSFJSKQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.254923645 (444h55m0.02556498s) [noise=true , address=45.91.95.3:4021 , piece_id=INUXGJNMLAZM6NI5PRUDW37MH42AMIC2V4EESX3NLMNZKZ33ZQWA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:26.394518518 (444h55m1.165159853s) [upload=355847]: Next returned nil segment Jul 11 22:05:26.394534878 (444h55m1.165176213s) [upload=355847]: Waiting for error group managing segments... Jul 11 22:05:26.700772093 (444h55m1.471413428s) [noise=true , address=79.127.205.233:4656 , piece_id=4YNKONSSBYFTV4CCGI3JKAWSADBEOBKKE24L7EEO64CWUZSFJSKQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:29.109783784 (444h55m3.880425109s) [noise=true , address=79.127.220.101:4059 , piece_id=5Y2CK5SM54UODEFCGPDTAAAMF3JRC5CV7AGQYCZ6PW47EQD5VV2Q, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:33.759174840 (444h55m8.529816165s) [noise=true , address=77.90.31.12:4619 , piece_id=V2CFGF4J3CRRMQVBVUTYFOAU6NHNW7ZZNXHLW3B5WC6RTW6MSJZQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:34.089773700 (444h55m8.860415035s) [noise=true , address=77.90.31.13:4640 , piece_id=DUQI7XZUGJFKCW5D73FZGEDSLR3S5PKCPPH6V2XLWSR35BUQZ54A, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:35.743647957 (444h55m10.514289292s) [noise=true , address=45.91.116.11:4149 , piece_id=JJ735AOMNAZ5V2BWJGA2BPLQ375574VANFAFBGXAOVNODKGD4K5Q, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:41.189380742 (444h55m15.960022077s) [noise=true , address=77.90.31.11:4588 , piece_id=QKDMMYO5KGDLTFB74BSPG7UYP6VWEUQBS52WPLW2VKDQHXM4LNOQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:41.645213384 (444h55m16.415854719s) [noise=true , address=77.90.31.21:4910 , piece_id=R43OAJYZUOW3ZNG2PWF7BMXRO37U44MFEAGPEAPQCZNUP6GUEBIQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:41.866575910 (444h55m16.637217245s) [noise=true , address=45.91.95.2:4021 , piece_id=37GWHZ27ALNE3XJWYMRZ3QDB6HYL6JL5KSSRMR3BQMBMDEMQ6NAA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:44.740074006 (444h55m19.510715341s) [noise=true , address=77.90.31.15:4707 , piece_id=IULW4EB4E6R56FTK3ZT527MYU22CQFW7XMWUC7ULUGJPS5VSL2ZA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:46.188108286 (444h55m20.958749621s) [noise=true , address=45.91.95.4:4021 , piece_id=NMZAYDJGGRU27TY7QCKYDGGFWBWMWWKHIROPT57QVUSC4HKAUHMQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:55.408676041 (444h55m30.179317376s) [noise=true , address=45.91.95.3:4021 , piece_id=INUXGJNMLAZM6NI5PRUDW37MH42AMIC2V4EESX3NLMNZKZ33ZQWA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:06:24.142005152 (444h55m58.912646487s) [noise=true , address=45.91.95.5:4034 , piece_id=CCLUI4ELT3AMG62RKTLXEGM4LBCQFTQQJHKKAD7LMOFE4XVYTWUA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:1gzELQ4cJ6iXuHuRJTUE1gSGj2Wz2VYoqLdZpEzey5RQRvGQv8, address:45.91.95.5:4034): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:06:24.142454010 (444h55m58.913095375s) [noise=true , address=104.36.82.3:4799 , piece_id=E3PJM326VZOUQQIXYYK2T3IDEDGTHYR6M2GNPZECR4YXJRSJGCTA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:104.36.82.3:4799): protocol: write tcp 188.166.225.170:46732->104.36.82.3:4799: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142530917 (444h55m58.913172282s) [noise=true , address=104.36.82.6:4901 , piece_id=SE4Y3VRZRUWDBBIQFYCRDH3KYSPWFSN4KOCS3KKUMLLTDRX7S3PA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:1SRXGomdNGRrsPBzD8YU34Y1EiuiJdnwuB79v8cMSLgTd7jpei, address:104.36.82.6:4901): protocol: write tcp 188.166.225.170:41510->104.36.82.6:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142582606 (444h55m58.913223991s) [noise=true , address=104.36.82.19:4414 , piece_id=4YUT5MJANGYRZUSAVBJ4NKZH5V5XTTMQI4OZX5WBZSMNS7IRJDPA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:1R6GJNS7bXfsDLsp3CE9qcvTbcT44QmgvSFzVbzpukyVYsezRQ, address:104.36.82.19:4414): protocol: write tcp 188.166.225.170:59368->104.36.82.19:4414: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142659403 (444h55m58.913300739s) [noise=true , address=104.36.82.4:4820 , piece_id=7DBP4GL3YMF23V3U27ZS6STMJWS2FLY4HWDKHBQL67A2C3SKUDWA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:12TnUNGbwHSf335XW7vNuq1ffwhFkgxomckBQmF189779tRU59k, address:104.36.82.4:4820): protocol: write tcp 188.166.225.170:57218->104.36.82.4:4820: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142696935 (444h55m58.91333832s) [noise=true , address=104.36.82.51:4081 , piece_id=YQKYMQJ676ODM3ZTSHRQANURNQB4WKQS7HW34AB4G35FVYYC5FAA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:121Gj3trM89mKpTffMYkrV9PrwkPqQU2LauCiG1v3J91Mej9KWF, address:104.36.82.51:4081): protocol: write tcp 188.166.225.170:43764->104.36.82.51:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142743154 (444h55m58.913384489s) [noise=true , address=104.36.82.50:4035 , piece_id=RPDFNBKOYV4HTINO32MZ5K7ESLNDTUPSAHXKLKJ2J7XWWV7RJGOQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:12nevdZspAt1ce9XA2Emh8QJDxcmEKpYemCMwWidVi8RU6T6vZD, address:104.36.82.50:4035): protocol: rpc: dial tcp 104.36.82.50:4035: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142774984 (444h55m58.91341635s) [seg_pos={0 0}, upload=355847]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Jul 11 22:06:24.374341223 (444h55m59.144982558s) [upload=355847]: Done uploading segments... Jul 11 22:06:24.374371692 (444h55m59.145013097s) [upload=355847]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 35.42363097s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....................................................XXXX.XXX] duration: 30.230574384s FAIL (2025-07-11 22:06:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== :18.381265190 (444h55m53.151906516s) [upload=355858]: Got next segment. Inline: false Jul 11 22:06:18.381337047 (444h55m53.151978463s) [upload=355858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:06:18.381347548 (444h55m53.151988883s) [upload=355858]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:06:18.614242919 (444h55m53.384884244s) [seg_pos={0 0}, upload=355858]: Begin upload segment... Jul 11 22:06:18.614305669 (444h55m53.384947004s) [seg_pos={0 0}, upload=355858]: Done begin upload segment. Jul 11 22:06:18.614333091 (444h55m53.384974426s) [noise=true , address=77.90.31.11:4553 , piece_id=EVRRYIRGMFPJAXWKUE2OOR7J2L3QGE347ANY2OOFWHNEH4UUBPGA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.614362016 (444h55m53.385003351s) [noise=true , address=104.36.82.6:4876 , piece_id=SSS74AGGR2HNXPMZVLKKK6UTNYE3K2T7TUD2DRLPBQPXTRPV6LFA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.614701386 (444h55m53.385342721s) [noise=true , address=45.91.116.7:4011 , piece_id=3EQRROT2UWX77IGGLZP372Y62XOOJJPPSIJVNKXHWEMQK6V4AFKQ, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.614771299 (444h55m53.385412635s) [noise=true , address=45.91.116.6:4975 , piece_id=DCMZ7C65ATFV7Z63V4I6N2KNRAGQ6Y5OPFY7VQRQBZDI47MQHVKA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.614984377 (444h55m53.385625712s) [noise=true , address=104.36.82.8:4972 , piece_id=AOHY37DBFVRMH6BK36AU7IHJLOAUEZ2KO5GDPJRBIHEC3OZWXJ2Q, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.615261137 (444h55m53.385902472s) [noise=true , address=104.36.82.12:4150 , piece_id=56LQFZEFONKFKCHPKWB77EKPCO3SN4Q4PUECPGKGM2CQ56VVGHSQ, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.615578604 (444h55m53.386219939s) [noise=true , address=77.90.31.12:4590 , piece_id=XEGXAIPX45GGZMUSU3YTLGPLTI3SMN6NU5YNMBRF5XIT7VU776SA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.615871434 (444h55m53.386512769s) [noise=true , address=45.91.116.11:4118 , piece_id=J7TMC6LCWVJ6LAEGFBKXVBQS4M47QDGW3KBOGX5NNZSEIY5YEXDQ, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.616136342 (444h55m53.386777667s) [noise=true , address=77.90.31.17:4770 , piece_id=QZYL36NDFWRBIPWRQM364VRCVCGDQGYYUNX6SBSJ7Y7WHGWYZ6QQ, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.616462004 (444h55m53.387103339s) [noise=true , address=77.90.31.15:4698 , piece_id=BHY2EPO4O7BZOYSTERORJLUWA2DFUYB3UI7GV2W77MFF4XSCXHYA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.616733854 (444h55m53.387375189s) [noise=true , address=104.36.82.15:4257 , piece_id=KYLQMKV7SUYWA4UYCCHSW33WWRJMEQN7MX6HZJ4WGW54BXUUGNMA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.616978993 (444h55m53.387620328s) [noise=true , address=104.36.82.9:4007 , piece_id=AEHVS2MZ45FO3ELZFGT7KJ2D52LYLKBQIESJ7X2K6KJR6EFYYLVQ, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.617199655 (444h55m53.38784099s) [noise=true , address=77.90.31.13:4626 , piece_id=5ELJOPR2OZKADAZLEYU2QYHEBLB54ZRVHHEOVUM2JJZLQKO2A5FA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.617467478 (444h55m53.388108813s) [noise=true , address=104.36.82.4:4802 , piece_id=M5CHHHR7OBQLHAUJ4LHNSDGVAD7DOQI5BEBYR4M5KHMSZRUSQLKA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.617726363 (444h55m53.388367698s) [noise=true , address=77.90.31.16:4733 , piece_id=OX5KQRXKQPTOFCKNVPSZEUVJ3SR3LXMPSOBQVXRJUKKYHPYKQVYA, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:18.617978385 (444h55m53.38861972s) [noise=true , address=104.36.82.16:4295 , piece_id=O64E72UWQDBEPHOFUJKPNWRSTDCNTIY25L5I6LG3CUQLP3WBWO7A, seg_pos={0 0}, upload=355858]: Uploading piece... Jul 11 22:06:23.118766585 (444h55m57.88940791s) [upload=355858]: Next returned nil segment Jul 11 22:06:23.118784489 (444h55m57.889425824s) [upload=355858]: Waiting for error group managing segments... Jul 11 22:06:24.424282938 (444h55m59.194924273s) [noise=true , address=77.90.31.12:4590 , piece_id=XEGXAIPX45GGZMUSU3YTLGPLTI3SMN6NU5YNMBRF5XIT7VU776SA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: <nil> Jul 11 22:06:24.930443063 (444h55m59.701084398s) [noise=true , address=45.91.116.6:4975 , piece_id=DCMZ7C65ATFV7Z63V4I6N2KNRAGQ6Y5OPFY7VQRQBZDI47MQHVKA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: <nil> Jul 11 22:06:32.507061142 (444h56m7.277702477s) [noise=true , address=77.90.31.17:4770 , piece_id=QZYL36NDFWRBIPWRQM364VRCVCGDQGYYUNX6SBSJ7Y7WHGWYZ6QQ, seg_pos={0 0}, upload=355858]: Done uploading piece. err: <nil> Jul 11 22:06:36.071825820 (444h56m10.842467155s) [noise=true , address=45.91.116.11:4118 , piece_id=J7TMC6LCWVJ6LAEGFBKXVBQS4M47QDGW3KBOGX5NNZSEIY5YEXDQ, seg_pos={0 0}, upload=355858]: Done uploading piece. err: <nil> Jul 11 22:06:38.006856823 (444h56m12.777498148s) [noise=true , address=77.90.31.15:4698 , piece_id=BHY2EPO4O7BZOYSTERORJLUWA2DFUYB3UI7GV2W77MFF4XSCXHYA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: <nil> Jul 11 22:06:39.890428867 (444h56m14.661070202s) [noise=true , address=45.91.116.7:4011 , piece_id=3EQRROT2UWX77IGGLZP372Y62XOOJJPPSIJVNKXHWEMQK6V4AFKQ, seg_pos={0 0}, upload=355858]: Done uploading piece. err: <nil> Jul 11 22:06:41.755616761 (444h56m16.526258096s) [noise=true , address=77.90.31.13:4626 , piece_id=5ELJOPR2OZKADAZLEYU2QYHEBLB54ZRVHHEOVUM2JJZLQKO2A5FA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: <nil> Jul 11 22:06:44.875860520 (444h56m19.646501855s) [noise=true , address=77.90.31.16:4733 , piece_id=OX5KQRXKQPTOFCKNVPSZEUVJ3SR3LXMPSOBQVXRJUKKYHPYKQVYA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: <nil> Jul 11 22:06:47.743461387 (444h56m22.514102722s) [noise=true , address=104.36.82.15:4257 , piece_id=KYLQMKV7SUYWA4UYCCHSW33WWRJMEQN7MX6HZJ4WGW54BXUUGNMA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:47.743525499 (444h56m22.514166824s) [noise=true , address=104.36.82.12:4150 , piece_id=56LQFZEFONKFKCHPKWB77EKPCO3SN4Q4PUECPGKGM2CQ56VVGHSQ, seg_pos={0 0}, upload=355858]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: rpc: dial tcp 104.36.82.12:4150: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:47.743534798 (444h56m22.514176133s) [noise=true , address=77.90.31.11:4553 , piece_id=EVRRYIRGMFPJAXWKUE2OOR7J2L3QGE347ANY2OOFWHNEH4UUBPGA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:35320->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:35320->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35320->77.90.31.11:4553: use of closed network connection Jul 11 22:06:47.743567200 (444h56m22.514208535s) [noise=true , address=104.36.82.4:4802 , piece_id=M5CHHHR7OBQLHAUJ4LHNSDGVAD7DOQI5BEBYR4M5KHMSZRUSQLKA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: rpc: dial tcp 104.36.82.4:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:47.743581677 (444h56m22.514223012s) [noise=true , address=104.36.82.6:4876 , piece_id=SSS74AGGR2HNXPMZVLKKK6UTNYE3K2T7TUD2DRLPBQPXTRPV6LFA, seg_pos={0 0}, upload=355858]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 188.166.225.170:36034->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:47.743612315 (444h56m22.51425366s) [noise=true , address=104.36.82.9:4007 , piece_id=AEHVS2MZ45FO3ELZFGT7KJ2D52LYLKBQIESJ7X2K6KJR6EFYYLVQ, seg_pos={0 0}, upload=355858]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:47.743622115 (444h56m22.51426345s) [noise=true , address=104.36.82.16:4295 , piece_id=O64E72UWQDBEPHOFUJKPNWRSTDCNTIY25L5I6LG3CUQLP3WBWO7A, seg_pos={0 0}, upload=355858]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 188.166.225.170:44608->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:47.743637303 (444h56m22.514278638s) [noise=true , address=104.36.82.8:4972 , piece_id=AOHY37DBFVRMH6BK36AU7IHJLOAUEZ2KO5GDPJRBIHEC3OZWXJ2Q, seg_pos={0 0}, upload=355858]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: rpc: dial tcp 104.36.82.8:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:47.743660047 (444h56m22.514301382s) [seg_pos={0 0}, upload=355858]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 11 22:06:47.972355192 (444h56m22.742996577s) [upload=355858]: Done uploading segments... Jul 11 22:06:47.972386271 (444h56m22.743027606s) [upload=355858]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.230589753s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XX.X.] duration: 30.761094906s FAIL (2025-07-11 22:06:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.761119855s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................................X.XXX.XX] duration: 30.001030339s FAIL (2025-07-11 22:06:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001094972s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................X..XX] duration: 1m0.233942188s FAIL (2025-07-11 22:06:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 47]: Done begin upload segment. Jul 11 22:05:25.250192172 (444h55m0.020833497s) [noise=true , address=104.36.82.3:4799 , piece_id=E3PJM326VZOUQQIXYYK2T3IDEDGTHYR6M2GNPZECR4YXJRSJGCTA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.250231017 (444h55m0.020872352s) [noise=true , address=77.90.31.11:4588 , piece_id=QKDMMYO5KGDLTFB74BSPG7UYP6VWEUQBS52WPLW2VKDQHXM4LNOQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.250569715 (444h55m0.02121105s) [noise=true , address=45.91.95.5:4034 , piece_id=CCLUI4ELT3AMG62RKTLXEGM4LBCQFTQQJHKKAD7LMOFE4XVYTWUA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.250766191 (444h55m0.021407526s) [noise=true , address=77.90.31.12:4619 , piece_id=V2CFGF4J3CRRMQVBVUTYFOAU6NHNW7ZZNXHLW3B5WC6RTW6MSJZQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.250847707 (444h55m0.021489042s) [noise=true , address=45.91.116.11:4149 , piece_id=JJ735AOMNAZ5V2BWJGA2BPLQ375574VANFAFBGXAOVNODKGD4K5Q, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251107715 (444h55m0.02174905s) [noise=true , address=104.36.82.19:4414 , piece_id=4YUT5MJANGYRZUSAVBJ4NKZH5V5XTTMQI4OZX5WBZSMNS7IRJDPA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251178610 (444h55m0.021819945s) [noise=true , address=79.127.220.101:4059 , piece_id=5Y2CK5SM54UODEFCGPDTAAAMF3JRC5CV7AGQYCZ6PW47EQD5VV2Q, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251489936 (444h55m0.022131341s) [noise=true , address=104.36.82.51:4081 , piece_id=YQKYMQJ676ODM3ZTSHRQANURNQB4WKQS7HW34AB4G35FVYYC5FAA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251521937 (444h55m0.022163272s) [noise=true , address=77.90.31.21:4910 , piece_id=R43OAJYZUOW3ZNG2PWF7BMXRO37U44MFEAGPEAPQCZNUP6GUEBIQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.251908417 (444h55m0.022549742s) [noise=true , address=77.90.31.13:4640 , piece_id=DUQI7XZUGJFKCW5D73FZGEDSLR3S5PKCPPH6V2XLWSR35BUQZ54A, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.252223779 (444h55m0.022865115s) [noise=true , address=104.36.82.6:4901 , piece_id=SE4Y3VRZRUWDBBIQFYCRDH3KYSPWFSN4KOCS3KKUMLLTDRX7S3PA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.253148930 (444h55m0.023790265s) [noise=true , address=45.91.95.2:4021 , piece_id=37GWHZ27ALNE3XJWYMRZ3QDB6HYL6JL5KSSRMR3BQMBMDEMQ6NAA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.253471086 (444h55m0.024112411s) [noise=true , address=45.91.95.4:4021 , piece_id=NMZAYDJGGRU27TY7QCKYDGGFWBWMWWKHIROPT57QVUSC4HKAUHMQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.253751101 (444h55m0.024392436s) [noise=true , address=104.36.82.50:4035 , piece_id=RPDFNBKOYV4HTINO32MZ5K7ESLNDTUPSAHXKLKJ2J7XWWV7RJGOQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.254046447 (444h55m0.024687802s) [noise=true , address=77.90.31.15:4707 , piece_id=IULW4EB4E6R56FTK3ZT527MYU22CQFW7XMWUC7ULUGJPS5VSL2ZA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.254400644 (444h55m0.025041979s) [noise=true , address=104.36.82.4:4820 , piece_id=7DBP4GL3YMF23V3U27ZS6STMJWS2FLY4HWDKHBQL67A2C3SKUDWA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.254676783 (444h55m0.025318108s) [noise=true , address=79.127.205.233:4656 , piece_id=4YNKONSSBYFTV4CCGI3JKAWSADBEOBKKE24L7EEO64CWUZSFJSKQ, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:25.254923645 (444h55m0.02556498s) [noise=true , address=45.91.95.3:4021 , piece_id=INUXGJNMLAZM6NI5PRUDW37MH42AMIC2V4EESX3NLMNZKZ33ZQWA, seg_pos={0 0}, upload=355847]: Uploading piece... Jul 11 22:05:26.394518518 (444h55m1.165159853s) [upload=355847]: Next returned nil segment Jul 11 22:05:26.394534878 (444h55m1.165176213s) [upload=355847]: Waiting for error group managing segments... Jul 11 22:05:26.700772093 (444h55m1.471413428s) [noise=true , address=79.127.205.233:4656 , piece_id=4YNKONSSBYFTV4CCGI3JKAWSADBEOBKKE24L7EEO64CWUZSFJSKQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:29.109783784 (444h55m3.880425109s) [noise=true , address=79.127.220.101:4059 , piece_id=5Y2CK5SM54UODEFCGPDTAAAMF3JRC5CV7AGQYCZ6PW47EQD5VV2Q, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:33.759174840 (444h55m8.529816165s) [noise=true , address=77.90.31.12:4619 , piece_id=V2CFGF4J3CRRMQVBVUTYFOAU6NHNW7ZZNXHLW3B5WC6RTW6MSJZQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:34.089773700 (444h55m8.860415035s) [noise=true , address=77.90.31.13:4640 , piece_id=DUQI7XZUGJFKCW5D73FZGEDSLR3S5PKCPPH6V2XLWSR35BUQZ54A, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:35.743647957 (444h55m10.514289292s) [noise=true , address=45.91.116.11:4149 , piece_id=JJ735AOMNAZ5V2BWJGA2BPLQ375574VANFAFBGXAOVNODKGD4K5Q, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:41.189380742 (444h55m15.960022077s) [noise=true , address=77.90.31.11:4588 , piece_id=QKDMMYO5KGDLTFB74BSPG7UYP6VWEUQBS52WPLW2VKDQHXM4LNOQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:41.645213384 (444h55m16.415854719s) [noise=true , address=77.90.31.21:4910 , piece_id=R43OAJYZUOW3ZNG2PWF7BMXRO37U44MFEAGPEAPQCZNUP6GUEBIQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:41.866575910 (444h55m16.637217245s) [noise=true , address=45.91.95.2:4021 , piece_id=37GWHZ27ALNE3XJWYMRZ3QDB6HYL6JL5KSSRMR3BQMBMDEMQ6NAA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:44.740074006 (444h55m19.510715341s) [noise=true , address=77.90.31.15:4707 , piece_id=IULW4EB4E6R56FTK3ZT527MYU22CQFW7XMWUC7ULUGJPS5VSL2ZA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:46.188108286 (444h55m20.958749621s) [noise=true , address=45.91.95.4:4021 , piece_id=NMZAYDJGGRU27TY7QCKYDGGFWBWMWWKHIROPT57QVUSC4HKAUHMQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:05:55.408676041 (444h55m30.179317376s) [noise=true , address=45.91.95.3:4021 , piece_id=INUXGJNMLAZM6NI5PRUDW37MH42AMIC2V4EESX3NLMNZKZ33ZQWA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: <nil> Jul 11 22:06:24.142005152 (444h55m58.912646487s) [noise=true , address=45.91.95.5:4034 , piece_id=CCLUI4ELT3AMG62RKTLXEGM4LBCQFTQQJHKKAD7LMOFE4XVYTWUA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:1gzELQ4cJ6iXuHuRJTUE1gSGj2Wz2VYoqLdZpEzey5RQRvGQv8, address:45.91.95.5:4034): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:06:24.142454010 (444h55m58.913095375s) [noise=true , address=104.36.82.3:4799 , piece_id=E3PJM326VZOUQQIXYYK2T3IDEDGTHYR6M2GNPZECR4YXJRSJGCTA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:104.36.82.3:4799): protocol: write tcp 188.166.225.170:46732->104.36.82.3:4799: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142530917 (444h55m58.913172282s) [noise=true , address=104.36.82.6:4901 , piece_id=SE4Y3VRZRUWDBBIQFYCRDH3KYSPWFSN4KOCS3KKUMLLTDRX7S3PA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:1SRXGomdNGRrsPBzD8YU34Y1EiuiJdnwuB79v8cMSLgTd7jpei, address:104.36.82.6:4901): protocol: write tcp 188.166.225.170:41510->104.36.82.6:4901: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142582606 (444h55m58.913223991s) [noise=true , address=104.36.82.19:4414 , piece_id=4YUT5MJANGYRZUSAVBJ4NKZH5V5XTTMQI4OZX5WBZSMNS7IRJDPA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:1R6GJNS7bXfsDLsp3CE9qcvTbcT44QmgvSFzVbzpukyVYsezRQ, address:104.36.82.19:4414): protocol: write tcp 188.166.225.170:59368->104.36.82.19:4414: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142659403 (444h55m58.913300739s) [noise=true , address=104.36.82.4:4820 , piece_id=7DBP4GL3YMF23V3U27ZS6STMJWS2FLY4HWDKHBQL67A2C3SKUDWA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:12TnUNGbwHSf335XW7vNuq1ffwhFkgxomckBQmF189779tRU59k, address:104.36.82.4:4820): protocol: write tcp 188.166.225.170:57218->104.36.82.4:4820: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142696935 (444h55m58.91333832s) [noise=true , address=104.36.82.51:4081 , piece_id=YQKYMQJ676ODM3ZTSHRQANURNQB4WKQS7HW34AB4G35FVYYC5FAA, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:121Gj3trM89mKpTffMYkrV9PrwkPqQU2LauCiG1v3J91Mej9KWF, address:104.36.82.51:4081): protocol: write tcp 188.166.225.170:43764->104.36.82.51:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142743154 (444h55m58.913384489s) [noise=true , address=104.36.82.50:4035 , piece_id=RPDFNBKOYV4HTINO32MZ5K7ESLNDTUPSAHXKLKJ2J7XWWV7RJGOQ, seg_pos={0 0}, upload=355847]: Done uploading piece. err: ecclient: upload failed (node:12nevdZspAt1ce9XA2Emh8QJDxcmEKpYemCMwWidVi8RU6T6vZD, address:104.36.82.50:4035): protocol: rpc: dial tcp 104.36.82.50:4035: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:24.142774984 (444h55m58.91341635s) [seg_pos={0 0}, upload=355847]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Jul 11 22:06:24.374341223 (444h55m59.144982558s) [upload=355847]: Done uploading segments... Jul 11 22:06:24.374371692 (444h55m59.145013097s) [upload=355847]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.233961073s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XXXX.XX] duration: 30.240901158s FAIL (2025-07-11 22:06:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== upload segment... Jul 11 22:05:48.202083429 (444h55m22.972724764s) [seg_pos={0 0}, upload=355851]: Done begin upload segment. Jul 11 22:05:48.202104129 (444h55m22.972745464s) [noise=true , address=104.36.82.6:4876 , piece_id=7CQBLHTGVG7MGUI65IQXJW6HH4LCKBGIFPSLQYYEMUE5N7PNAUUA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.202148182 (444h55m22.972789517s) [noise=true , address=104.36.82.9:4007 , piece_id=7IDQZPAX4X3OP7JSRVTHQ6MM42ILENXUDQKLG7MT4WJSAYD5BZ4Q, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.202492442 (444h55m22.973133777s) [noise=true , address=104.36.82.10:4079 , piece_id=AEF5GCBWDYNPGAOI65S5RIG5DTB3NWXXHUJ5AN6AE5KR6J6MIWLQ, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.202673578 (444h55m22.973314913s) [noise=true , address=45.91.116.10:4081 , piece_id=6Q5BIXK5Q7NWOW3FWZXQRE5KWS5B3BWAILL4EDX7WKI6JB7N242A, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.202757558 (444h55m22.973398893s) [noise=true , address=77.90.31.11:4555 , piece_id=Z6SD6BP7JLQGE6F6AQVKRORM6L4EH6LXXZX5IDYTU4BMBGGS5HTA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.202997117 (444h55m22.973638452s) [noise=true , address=104.36.82.13:4187 , piece_id=RFZIKEBYFSA2WKK46L6X7B6MJFFB4FG4NIM4IRW7IHBDZEWJ7ESA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.203075507 (444h55m22.973716842s) [noise=true , address=104.36.82.11:4114 , piece_id=N2GH3MUE3CKLMF2IVB6VL4DVAEYIBTP5HBZWDCZXIAQLC67GRKSA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.203238098 (444h55m22.973879433s) [noise=true , address=77.90.31.16:4733 , piece_id=LEAUATY3SN6SVNVZNBHUE4VJTG36GOAUXUZ4NQARKCFEF22ONYOA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.203373738 (444h55m22.974015073s) [noise=true , address=104.36.82.10:4078 , piece_id=B7YOO2HKX2DSUPEP3CLR64MSOJMXALHX72P3X7Z2ITNLTZNNGOAA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.203778982 (444h55m22.974420317s) [noise=true , address=104.36.82.3:4767 , piece_id=7KYSC6ZYGQT44WR3KL6BUXJJPOIGFZVDEG6KA57E3TG6VZQK3IVQ, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.204130154 (444h55m22.974771489s) [noise=true , address=77.90.31.10:4519 , piece_id=JMVUHCKYW6PEWA3Z3TBHKC2N6UXYT5FGDBRFI2PBEFEFHDQYR2NQ, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.204352189 (444h55m22.974993524s) [noise=true , address=104.36.82.12:4151 , piece_id=XXIAP5QEW2GBTUTHOWL5CZJSO7MQKU7T55TW6ZRAZWWZCS22JEMQ, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.204602748 (444h55m22.975244083s) [noise=true , address=77.90.31.16:4735 , piece_id=MT367R44DLRBMHM7NK7L4GK4SHDU3CLKL4DDFCFN3ZWWNVAECEGA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.204926147 (444h55m22.975567482s) [noise=true , address=77.90.31.16:4734 , piece_id=SU4FU3ZUSA4AEL3AZ2SDEFI4M5OBAMOP2UV3IS7E2VQVGJ4W24ZA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.205145126 (444h55m22.975786461s) [noise=true , address=104.36.82.8:4970 , piece_id=AM4CNJGLD6XVQDGPKNYSWYVXILKWDZARBRKSTYL4VDU53WR47MCA, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:48.205339729 (444h55m22.975981064s) [noise=true , address=104.36.82.16:4295 , piece_id=32PDOAQWAX4VYF37AQQNCG3KB2PRXURPHCB5APZ4GQBK7ZZOAKXQ, seg_pos={0 0}, upload=355851]: Uploading piece... Jul 11 22:05:59.357347798 (444h55m34.127989133s) [upload=355851]: Next returned nil segment Jul 11 22:05:59.357364349 (444h55m34.128005684s) [upload=355851]: Waiting for error group managing segments... Jul 11 22:06:02.380894411 (444h55m37.151535746s) [noise=true , address=77.90.31.10:4519 , piece_id=JMVUHCKYW6PEWA3Z3TBHKC2N6UXYT5FGDBRFI2PBEFEFHDQYR2NQ, seg_pos={0 0}, upload=355851]: Done uploading piece. err: <nil> Jul 11 22:06:06.571948624 (444h55m41.342589959s) [noise=true , address=77.90.31.16:4735 , piece_id=MT367R44DLRBMHM7NK7L4GK4SHDU3CLKL4DDFCFN3ZWWNVAECEGA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: <nil> Jul 11 22:06:14.894353595 (444h55m49.66499493s) [noise=true , address=45.91.116.10:4081 , piece_id=6Q5BIXK5Q7NWOW3FWZXQRE5KWS5B3BWAILL4EDX7WKI6JB7N242A, seg_pos={0 0}, upload=355851]: Done uploading piece. err: <nil> Jul 11 22:06:15.469842396 (444h55m50.240483731s) [noise=true , address=77.90.31.16:4733 , piece_id=LEAUATY3SN6SVNVZNBHUE4VJTG36GOAUXUZ4NQARKCFEF22ONYOA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: <nil> Jul 11 22:06:17.240440124 (444h55m52.011081459s) [noise=true , address=77.90.31.11:4555 , piece_id=Z6SD6BP7JLQGE6F6AQVKRORM6L4EH6LXXZX5IDYTU4BMBGGS5HTA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: <nil> Jul 11 22:06:17.270687561 (444h55m52.041328896s) [noise=true , address=77.90.31.16:4734 , piece_id=SU4FU3ZUSA4AEL3AZ2SDEFI4M5OBAMOP2UV3IS7E2VQVGJ4W24ZA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:59710->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:59710->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59710->77.90.31.16:4734: use of closed network connection Jul 11 22:06:17.271618042 (444h55m52.042259367s) [noise=true , address=104.36.82.12:4151 , piece_id=XXIAP5QEW2GBTUTHOWL5CZJSO7MQKU7T55TW6ZRAZWWZCS22JEMQ, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:17.271650103 (444h55m52.042291438s) [noise=true , address=104.36.82.16:4295 , piece_id=32PDOAQWAX4VYF37AQQNCG3KB2PRXURPHCB5APZ4GQBK7ZZOAKXQ, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: rpc: dial tcp 104.36.82.16:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:17.271677275 (444h55m52.04231861s) [noise=true , address=104.36.82.6:4876 , piece_id=7CQBLHTGVG7MGUI65IQXJW6HH4LCKBGIFPSLQYYEMUE5N7PNAUUA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:17.272382875 (444h55m52.04302421s) [noise=true , address=104.36.82.9:4007 , piece_id=7IDQZPAX4X3OP7JSRVTHQ6MM42ILENXUDQKLG7MT4WJSAYD5BZ4Q, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 188.166.225.170:44790->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:17.272436328 (444h55m52.043077653s) [noise=true , address=104.36.82.10:4079 , piece_id=AEF5GCBWDYNPGAOI65S5RIG5DTB3NWXXHUJ5AN6AE5KR6J6MIWLQ, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:17.272474911 (444h55m52.043116246s) [noise=true , address=104.36.82.8:4970 , piece_id=AM4CNJGLD6XVQDGPKNYSWYVXILKWDZARBRKSTYL4VDU53WR47MCA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:17.273917992 (444h55m52.044559327s) [noise=true , address=104.36.82.13:4187 , piece_id=RFZIKEBYFSA2WKK46L6X7B6MJFFB4FG4NIM4IRW7IHBDZEWJ7ESA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 11 22:06:17.274420603 (444h55m52.045061938s) [noise=true , address=104.36.82.3:4767 , piece_id=7KYSC6ZYGQT44WR3KL6BUXJJPOIGFZVDEG6KA57E3TG6VZQK3IVQ, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: context deadline exceeded; context deadline exceeded Jul 11 22:06:17.274481881 (444h55m52.045123216s) [noise=true , address=104.36.82.11:4114 , piece_id=N2GH3MUE3CKLMF2IVB6VL4DVAEYIBTP5HBZWDCZXIAQLC67GRKSA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: context deadline exceeded; context deadline exceeded Jul 11 22:06:17.274528870 (444h55m52.045170205s) [noise=true , address=104.36.82.10:4078 , piece_id=B7YOO2HKX2DSUPEP3CLR64MSOJMXALHX72P3X7Z2ITNLTZNNGOAA, seg_pos={0 0}, upload=355851]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 11 22:06:17.274540242 (444h55m52.045181577s) [seg_pos={0 0}, upload=355851]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 11 22:06:17.507017964 (444h55m52.277659309s) [upload=355851]: Done uploading segments... Jul 11 22:06:17.507043173 (444h55m52.277684508s) [upload=355851]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.240927157s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XX.X] duration: 1m0.001472884s FAIL (2025-07-11 22:05:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001494215s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XXXX.X] duration: 30.234199671s FAIL (2025-07-11 22:05:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== Jul 11 22:05:16.177444009 (444h54m50.948085344s) [noise=true , address=104.36.82.9:4006 , piece_id=FRZJQCWY6NR76H4FALSCM4DKAEJO2BZRS3LZEAOR4BQSYX7OLBAA, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.177452054 (444h54m50.94809339s) [noise=true , address=104.36.82.4:4804 , piece_id=QV6FWWXTTDA6FK2A5OVVOE3KGOBPOY46VXBGYYFEDPLV7OJAIDUQ, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.177813586 (444h54m50.948454911s) [noise=true , address=77.90.31.11:4555 , piece_id=PTGGEI4CDJF2M2CXRRNVCKAQJBO53CDUU7VESCVX2BUXB5IBZRFA, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.178077121 (444h54m50.948718456s) [noise=true , address=77.90.31.14:4662 , piece_id=YV6NLZEMBOQW6X3JN7HZV22SMO4S4OM52SHB4IZTGSNDRORM2TMQ, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.178371353 (444h54m50.949012688s) [noise=true , address=45.91.116.4:4939 , piece_id=OUUQCLU2PP24JZZ26FMS5HA34MY7PCDDTWGL2VM5FKGZ27XPFX6Q, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.178449683 (444h54m50.949091018s) [noise=true , address=104.36.82.7:4910 , piece_id=3LUJDDRXBB65QE7GQZ23R3JFN3L4OGSKONFPAK2RMSDOZLUE7LDA, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.178718497 (444h54m50.949359833s) [noise=true , address=45.91.116.7:4011 , piece_id=TFDHRISM4ZPSMZFHL5BFQRYYAB4GBHCUBVGAC7JK2KDAOXRSZ2FA, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.178977002 (444h54m50.949618337s) [noise=true , address=104.36.82.8:4971 , piece_id=GX2SZZXH5BC3APSQ2TIRPRQCIGSX7IILBR7F4366A7WNE4K4BMRA, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.179159842 (444h54m50.949801177s) [noise=true , address=77.90.31.14:4663 , piece_id=ZEUWYFNLPLL4NE4GJRRJTAOBIG6TSXLFFCEETHZEET6MWG6C2UYQ, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.179416312 (444h54m50.950057647s) [noise=true , address=45.91.116.8:4045 , piece_id=6NLDN7Z3H2ZZAHAA45BPHQPG2X2FGI5JQIZAZFWUAKYYXOK2IM7Q, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.179614090 (444h54m50.950255426s) [noise=true , address=104.36.82.13:4185 , piece_id=DHA3FU6Y3KAHIXW53OBMZAT6MIQSWANIQKJSMOLDW4BDQKBPEQOQ, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.179678554 (444h54m50.95031989s) [noise=true , address=77.90.31.21:4877 , piece_id=Z2RKNF6QQLF7XSXXFR6BFF3I4ZETWLA5C54A2W5CNIQDZEG5LLRQ, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.179934063 (444h54m50.950575398s) [noise=true , address=77.90.31.11:4553 , piece_id=ZUBGSA7OKJDEEV5COIM7WTZMGTMXROESW4FQ5I5UK2THPUQJHCUQ, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.180085402 (444h54m50.950726737s) [noise=true , address=104.36.82.11:4115 , piece_id=YKYU3X66Z6Z3AK3GOG5LVDSL5VVOA56ZZVEDH7FMM2KHI6YUDM5A, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.180451954 (444h54m50.951093289s) [noise=true , address=77.90.31.15:4699 , piece_id=F5WPJ5CPQ3FTYU2CME6EU4JFXP534SRB2Z4Z6AWY2JEWRV4IJKIQ, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:16.180722582 (444h54m50.951363917s) [noise=true , address=45.91.116.9:4262 , piece_id=DS2PHZHHYHHCNYFSZW4KLQNANZEJ3LIBVRV72PMQ65ISS4YCCE3Q, seg_pos={0 0}, upload=355845]: Uploading piece... Jul 11 22:05:25.167649768 (444h54m59.938291103s) [upload=355845]: Next returned nil segment Jul 11 22:05:25.167671520 (444h54m59.938312845s) [upload=355845]: Waiting for error group managing segments... Jul 11 22:05:28.260748713 (444h55m3.031390078s) [noise=true , address=45.91.116.8:4045 , piece_id=6NLDN7Z3H2ZZAHAA45BPHQPG2X2FGI5JQIZAZFWUAKYYXOK2IM7Q, seg_pos={0 0}, upload=355845]: Done uploading piece. err: <nil> Jul 11 22:05:34.764669387 (444h55m9.535310723s) [noise=true , address=77.90.31.14:4663 , piece_id=ZEUWYFNLPLL4NE4GJRRJTAOBIG6TSXLFFCEETHZEET6MWG6C2UYQ, seg_pos={0 0}, upload=355845]: Done uploading piece. err: <nil> Jul 11 22:05:36.204047214 (444h55m10.974688539s) [noise=true , address=77.90.31.14:4662 , piece_id=YV6NLZEMBOQW6X3JN7HZV22SMO4S4OM52SHB4IZTGSNDRORM2TMQ, seg_pos={0 0}, upload=355845]: Done uploading piece. err: <nil> Jul 11 22:05:38.277270296 (444h55m13.047911631s) [noise=true , address=45.91.116.9:4262 , piece_id=DS2PHZHHYHHCNYFSZW4KLQNANZEJ3LIBVRV72PMQ65ISS4YCCE3Q, seg_pos={0 0}, upload=355845]: Done uploading piece. err: <nil> Jul 11 22:05:40.877199627 (444h55m15.647840962s) [noise=true , address=77.90.31.11:4555 , piece_id=PTGGEI4CDJF2M2CXRRNVCKAQJBO53CDUU7VESCVX2BUXB5IBZRFA, seg_pos={0 0}, upload=355845]: Done uploading piece. err: <nil> Jul 11 22:05:40.983719584 (444h55m15.75436092s) [noise=true , address=77.90.31.11:4553 , piece_id=ZUBGSA7OKJDEEV5COIM7WTZMGTMXROESW4FQ5I5UK2THPUQJHCUQ, seg_pos={0 0}, upload=355845]: Done uploading piece. err: <nil> Jul 11 22:05:43.501987015 (444h55m18.27262834s) [noise=true , address=45.91.116.4:4939 , piece_id=OUUQCLU2PP24JZZ26FMS5HA34MY7PCDDTWGL2VM5FKGZ27XPFX6Q, seg_pos={0 0}, upload=355845]: Done uploading piece. err: <nil> Jul 11 22:05:45.288528606 (444h55m20.059169941s) [noise=true , address=45.91.116.7:4011 , piece_id=TFDHRISM4ZPSMZFHL5BFQRYYAB4GBHCUBVGAC7JK2KDAOXRSZ2FA, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:37310->45.91.116.7:4011: use of closed network connection; write tcp 188.166.225.170:37310->45.91.116.7:4011: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37310->45.91.116.7:4011: use of closed network connection Jul 11 22:05:45.288649667 (444h55m20.059291012s) [noise=true , address=104.36.82.11:4115 , piece_id=YKYU3X66Z6Z3AK3GOG5LVDSL5VVOA56ZZVEDH7FMM2KHI6YUDM5A, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:45.288703780 (444h55m20.059345126s) [noise=true , address=104.36.82.13:4185 , piece_id=DHA3FU6Y3KAHIXW53OBMZAT6MIQSWANIQKJSMOLDW4BDQKBPEQOQ, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: rpc: dial tcp 104.36.82.13:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:45.288777522 (444h55m20.059418847s) [noise=true , address=77.90.31.21:4877 , piece_id=Z2RKNF6QQLF7XSXXFR6BFF3I4ZETWLA5C54A2W5CNIQDZEG5LLRQ, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:34002->77.90.31.21:4877: use of closed network connection; write tcp 188.166.225.170:34002->77.90.31.21:4877: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34002->77.90.31.21:4877: use of closed network connection Jul 11 22:05:45.288834841 (444h55m20.059476197s) [noise=true , address=104.36.82.4:4804 , piece_id=QV6FWWXTTDA6FK2A5OVVOE3KGOBPOY46VXBGYYFEDPLV7OJAIDUQ, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: rpc: dial tcp 104.36.82.4:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:45.288882393 (444h55m20.059523728s) [noise=true , address=104.36.82.9:4006 , piece_id=FRZJQCWY6NR76H4FALSCM4DKAEJO2BZRS3LZEAOR4BQSYX7OLBAA, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: rpc: dial tcp 104.36.82.9:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:45.288941095 (444h55m20.05958243s) [noise=true , address=77.90.31.15:4699 , piece_id=F5WPJ5CPQ3FTYU2CME6EU4JFXP534SRB2Z4Z6AWY2JEWRV4IJKIQ, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:41986->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:41986->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41986->77.90.31.15:4699: use of closed network connection Jul 11 22:05:45.288982754 (444h55m20.059624089s) [noise=true , address=104.36.82.7:4910 , piece_id=3LUJDDRXBB65QE7GQZ23R3JFN3L4OGSKONFPAK2RMSDOZLUE7LDA, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:45.289027841 (444h55m20.059669176s) [noise=true , address=104.36.82.8:4971 , piece_id=GX2SZZXH5BC3APSQ2TIRPRQCIGSX7IILBR7F4366A7WNE4K4BMRA, seg_pos={0 0}, upload=355845]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 188.166.225.170:49272->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:45.289046056 (444h55m20.059687391s) [seg_pos={0 0}, upload=355845]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 11 22:05:45.519757868 (444h55m20.290399203s) [upload=355845]: Done uploading segments... Jul 11 22:05:45.519785641 (444h55m20.290426986s) [upload=355845]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.234223095s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................................................X.XXX.X] duration: 30.001607883s FAIL (2025-07-11 22:05:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001651256s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X..X] duration: 1m0.003175017s FAIL (2025-07-11 22:05:23 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== (444h53m59.339085537s) [upload=355835]: Uploading segments... Jul 11 22:04:24.568536779 (444h53m59.339178114s) [upload=355835]: Got next segment. Inline: false Jul 11 22:04:24.568611411 (444h53m59.339252746s) [upload=355835]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:04:24.568616351 (444h53m59.339257686s) [upload=355835]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:04:25.046022424 (444h53m59.816663759s) [seg_pos={0 0}, upload=355835]: Begin upload segment... Jul 11 22:04:25.046064676 (444h53m59.816706011s) [seg_pos={0 0}, upload=355835]: Done begin upload segment. Jul 11 22:04:25.046085935 (444h53m59.81672727s) [noise=true , address=45.91.95.3:4003 , piece_id=V3TVSE2ARLI2L6ZDC3SUNKIPNWLN3N75NCJ6U74XS3RG73OTCCYA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.046110122 (444h53m59.816751457s) [noise=true , address=77.90.31.11:4573 , piece_id=3MC6SZTTBWDJUSQ5ZY4LEM7ZIUTASDQSZRNW5EECL4DRFITRZFWA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.046436146 (444h53m59.817077481s) [noise=true , address=104.36.82.19:4402 , piece_id=LKWJ3IQYO5C7AD6MJBDP36RYYPKD5VWOSVXD7LMUFOA3AUU4ISRA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.046696574 (444h53m59.817337899s) [noise=true , address=104.36.82.2:4763 , piece_id=ZCMXPFVWL3N5X3NWGBFLH2KKDUDN3QPX4HTHRWOZIWH3276J4HRA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.046939709 (444h53m59.817581044s) [noise=true , address=77.90.31.14:4670 , piece_id=FZIVROVRHP7UAMQTTDOTRHCVVW4FUMTCSLD3Q5CGSUKOKASUNKCQ, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.047155121 (444h53m59.817796456s) [noise=true , address=77.90.31.18:4834 , piece_id=ZQGSGR4FSJVWFMCXJIXAI76VKMTOMEWJW5BY3WXCKWDNU3WJM5EA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.047214935 (444h53m59.817856271s) [noise=true , address=79.127.219.38:4028 , piece_id=7RS64ITVCS27P2ZUGBWRIPJTZVF2QZBZTXU6YGZ7NNXR6LPLSHHQ, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.047482818 (444h53m59.818124153s) [noise=true , address=104.36.82.44:4896 , piece_id=5UED2MZNZS2PS6XWKPJGNJZQFLVXLOSTRTPRKAMV6UDZRE2RIL4A, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.047741082 (444h53m59.818382417s) [noise=true , address=45.91.116.8:4076 , piece_id=IXKMMCPXNRWTZ47KATCUZRZM7BO6MMLRROMZBTJJOIXDMFDMGNYA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.047938299 (444h53m59.818579634s) [noise=true , address=77.90.31.17:4774 , piece_id=6O5G6GCYAYDLTEZFYKK3Y2GQUBCZZAXNMEW4AWG75HWO3CMGJCAA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.048141889 (444h53m59.818783224s) [noise=true , address=104.36.82.35:4591 , piece_id=A42GHL2SK36YDEB3R6TS4RMAZK3EQPEA2S26CQ5C5KB4X4EN3TTA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.048308658 (444h53m59.818949993s) [noise=true , address=77.90.31.16:4757 , piece_id=GK4PGHNBZLZNQKZXUOV5UAAO3GINUFTQL7N6WZ6QUXFLLUN5RU7Q, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.048531945 (444h53m59.81917327s) [noise=true , address=45.91.95.5:4032 , piece_id=WNHBJXDRJ5AIS2YXJQ6NR5Q4T2NNCDQKHBH5OGUMXXAFVILHGTSA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.048753369 (444h53m59.819394704s) [noise=true , address=77.90.31.12:4615 , piece_id=FN4X4ZBTE32C6QVLFNM2HLLJXL6UJMUNVMQZPW6TIB6EB7OBHHCQ, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.049019798 (444h53m59.819661133s) [noise=true , address=104.36.82.36:4644 , piece_id=JP5WYM577P3BDPQPNZRP4EBQFWF4M5BKBNZEN6ZNMAIQALIT75RQ, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.049169806 (444h53m59.819811141s) [noise=true , address=45.91.95.4:4018 , piece_id=CMRT34JWFNIU7RE7FDLEMJUO5FHRWC6GLHGUL3WY3XJOFRBNG5YA, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.049360491 (444h53m59.820001826s) [noise=true , address=45.91.95.5:4016 , piece_id=U2TE6ECD3QLWIJSTEOLTF77K7VHJRWH2XFFW7ROZU3ZUELJGPOUQ, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:25.049578367 (444h53m59.820219703s) [noise=true , address=156.146.43.225:4433 , piece_id=QX34PM63X6JBDEVM6FR75KCBDHLYJRJKC6HDK6TG5OPCC4KTHCGQ, seg_pos={0 0}, upload=355835]: Uploading piece... Jul 11 22:04:26.544812586 (444h54m1.315453921s) [upload=355835]: Next returned nil segment Jul 11 22:04:26.544834037 (444h54m1.315475372s) [upload=355835]: Waiting for error group managing segments... Jul 11 22:04:26.966360045 (444h54m1.73700137s) [noise=true , address=104.36.82.35:4591 , piece_id=A42GHL2SK36YDEB3R6TS4RMAZK3EQPEA2S26CQ5C5KB4X4EN3TTA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:26.967880343 (444h54m1.738521678s) [noise=true , address=156.146.43.225:4433 , piece_id=QX34PM63X6JBDEVM6FR75KCBDHLYJRJKC6HDK6TG5OPCC4KTHCGQ, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:26.974333780 (444h54m1.744975115s) [noise=true , address=104.36.82.19:4402 , piece_id=LKWJ3IQYO5C7AD6MJBDP36RYYPKD5VWOSVXD7LMUFOA3AUU4ISRA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:28.207984403 (444h54m2.978625778s) [noise=true , address=104.36.82.44:4896 , piece_id=5UED2MZNZS2PS6XWKPJGNJZQFLVXLOSTRTPRKAMV6UDZRE2RIL4A, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:28.878263138 (444h54m3.648904473s) [noise=true , address=104.36.82.2:4763 , piece_id=ZCMXPFVWL3N5X3NWGBFLH2KKDUDN3QPX4HTHRWOZIWH3276J4HRA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:29.419525667 (444h54m4.190167002s) [noise=true , address=79.127.219.38:4028 , piece_id=7RS64ITVCS27P2ZUGBWRIPJTZVF2QZBZTXU6YGZ7NNXR6LPLSHHQ, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:30.486226571 (444h54m5.256867906s) [noise=true , address=104.36.82.36:4644 , piece_id=JP5WYM577P3BDPQPNZRP4EBQFWF4M5BKBNZEN6ZNMAIQALIT75RQ, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:35.729201426 (444h54m10.499842761s) [noise=true , address=77.90.31.16:4757 , piece_id=GK4PGHNBZLZNQKZXUOV5UAAO3GINUFTQL7N6WZ6QUXFLLUN5RU7Q, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:39.210822849 (444h54m13.981464184s) [noise=true , address=77.90.31.11:4573 , piece_id=3MC6SZTTBWDJUSQ5ZY4LEM7ZIUTASDQSZRNW5EECL4DRFITRZFWA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:43.214044927 (444h54m17.984686262s) [noise=true , address=77.90.31.17:4774 , piece_id=6O5G6GCYAYDLTEZFYKK3Y2GQUBCZZAXNMEW4AWG75HWO3CMGJCAA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:48.465578324 (444h54m23.236219659s) [noise=true , address=77.90.31.18:4834 , piece_id=ZQGSGR4FSJVWFMCXJIXAI76VKMTOMEWJW5BY3WXCKWDNU3WJM5EA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:53.158286728 (444h54m27.928928063s) [noise=true , address=77.90.31.14:4670 , piece_id=FZIVROVRHP7UAMQTTDOTRHCVVW4FUMTCSLD3Q5CGSUKOKASUNKCQ, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:04:54.967072216 (444h54m29.737713541s) [noise=true , address=77.90.31.12:4615 , piece_id=FN4X4ZBTE32C6QVLFNM2HLLJXL6UJMUNVMQZPW6TIB6EB7OBHHCQ, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:05:19.358645167 (444h54m54.129286502s) [noise=true , address=45.91.116.8:4076 , piece_id=IXKMMCPXNRWTZ47KATCUZRZM7BO6MMLRROMZBTJJOIXDMFDMGNYA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: <nil> Jul 11 22:05:19.358667811 (444h54m54.129309146s) [seg_pos={0 0}, upload=355835]: Segment reached optimal threshold of 14 pieces. Jul 11 22:05:19.358744488 (444h54m54.129385823s) [noise=true , address=45.91.95.5:4032 , piece_id=WNHBJXDRJ5AIS2YXJQ6NR5Q4T2NNCDQKHBH5OGUMXXAFVILHGTSA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V3ESTUgtnEfJjDyUqT4kG6AjdyH61tYAuRb4spQev1T52fDfj): protocol: expected piece hash; context canceled; EOF Jul 11 22:05:19.358780787 (444h54m54.129422112s) [noise=true , address=45.91.95.4:4018 , piece_id=CMRT34JWFNIU7RE7FDLEMJUO5FHRWC6GLHGUL3WY3XJOFRBNG5YA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YG8X6frBEE1J6S7Uubd8Ca2jQDjdcNToPSVBxW8uXFxwPbSPs): protocol: expected piece hash; context canceled; EOF Jul 11 22:05:19.358826114 (444h54m54.129467439s) [noise=true , address=45.91.95.5:4016 , piece_id=U2TE6ECD3QLWIJSTEOLTF77K7VHJRWH2XFFW7ROZU3ZUELJGPOUQ, seg_pos={0 0}, upload=355835]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16f3nMiQEcR88RtL6XErp3hjndPrsJ6VnphVf1Ai4uz88Q66ko): protocol: expected piece hash; context canceled; EOF Jul 11 22:05:19.358917969 (444h54m54.129559305s) [noise=true , address=45.91.95.3:4003 , piece_id=V3TVSE2ARLI2L6ZDC3SUNKIPNWLN3N75NCJ6U74XS3RG73OTCCYA, seg_pos={0 0}, upload=355835]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12775mgnBsA3595WztdcNsUBVWCLeeZECij1L2EnwuvzCCtMouk): protocol: write tcp 188.166.225.170:57746->45.91.95.3:4003: use of closed network connection; write tcp 188.166.225.170:57746->45.91.95.3:4003: use of closed network connection Jul 11 22:05:19.358934441 (444h54m54.129575776s) [seg_pos={0 0}, upload=355835]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 11 22:05:19.359009735 (444h54m54.12965106s) [upload=355835]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 11 22:05:19.359014004 (444h54m54.129655339s) [upload=355835]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 11 22:05:19.661247308 (444h54m54.431888643s) [upload=355835]: Done uploading segments... Jul 11 22:05:19.661263249 (444h54m54.431904584s) [upload=355835]: Upload finished. err: <nil> Duration: 1m0.003202149s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXXX.] duration: 21.404184101s FAIL (2025-07-11 22:04:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 63 , piece_id=DXLYEPO3MK6SC3FNCJ7XGIXAXXSTFQ6SCKMC7I27EEZCDAOYPGVA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.588324207 (444h53m11.358965542s) [noise=true , address=45.91.116.10:4082 , piece_id=2BW7GEGPGC5QTGNP3YWLG5ITLZ4ZAFKO4WK2UG2VMHNLAEIOJYBQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.588630102 (444h53m11.359271457s) [noise=true , address=45.91.116.7:4010 , piece_id=BHJ2V4SGO25ECZIEDRIFEJVNAP6JK4AJ46HQA64ARTMCXVEARWGQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.588716027 (444h53m11.359357362s) [noise=true , address=104.36.82.14:4221 , piece_id=3AQIO6ZO6FF5H4C4KYGIEXVLRMGGSA7MO64KQT2FXOJQL3366XCA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.588939384 (444h53m11.359580719s) [noise=true , address=104.36.82.16:4293 , piece_id=RXEYWHR457PJV2TBV6IIFEM4JQUTBRI5XYPDFN637FJ5CSY2XLVA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589064544 (444h53m11.359705869s) [noise=true , address=45.91.116.8:4047 , piece_id=ABQ3R2LKGVF4FC2JN3ELYI7QNRKUIBAWRQ4DRLVAYKKJLHT7KRJA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589149476 (444h53m11.359790811s) [noise=true , address=104.36.82.2:4732 , piece_id=OZBTQI47TOFIN532E6SQFE2ES7AH2JIUI4TYVXSJQDDRSGK352AA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589406508 (444h53m11.360047843s) [noise=true , address=104.36.82.11:4115 , piece_id=6Z43WIGHE6XZYF5RVJ7H4OP5SUGJEUQB6AT7CYPJSI2WF7L2ORUA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589548579 (444h53m11.360189914s) [noise=true , address=104.36.82.15:4259 , piece_id=FSTG7RKNERGMO5DQG3XY4L6WIOOAPW46FLZVENOLJ4BXU4Q3HXFQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589679801 (444h53m11.360321136s) [noise=true , address=45.91.116.13:4153 , piece_id=VAVXHRZ6D63GXY7AR6YKJDRH3HLUFSDDWQBBJCXOPCLJ6AKVJ4DQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589939458 (444h53m11.360580793s) [noise=true , address=104.36.82.7:4909 , piece_id=3A5D2ZOUIKPV7D34I3IYAKCAZZXEVR7V4NMMGYFRFIUAIUG45EWQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.590036683 (444h53m11.360678019s) [noise=true , address=45.91.116.11:4119 , piece_id=RNPSA3BWL4BVYCH6WIPK54WZMP5IZ35LNZQRGQCZ5FLXH43V6WNA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.594031947 (444h53m11.364673282s) [noise=true , address=104.36.82.13:4186 , piece_id=2XLPJQOI3KB2XAY2TWJD6JGFZPL65366HKT4S7MGQEESZUSU3L7A, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.594404540 (444h53m11.365045875s) [noise=true , address=77.90.31.12:4589 , piece_id=ZVPSEXXLSZ3ZC5IOOG26XIU2RR3LPH7QFYTA4PQC45YZ4HHYW7PQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:39.469032713 (444h53m14.239674038s) [upload=355823]: Next returned nil segment Jul 11 22:03:39.469049616 (444h53m14.239690941s) [upload=355823]: Waiting for error group managing segments... Jul 11 22:03:40.688983051 (444h53m15.459624386s) [noise=true , address=45.91.116.11:4119 , piece_id=RNPSA3BWL4BVYCH6WIPK54WZMP5IZ35LNZQRGQCZ5FLXH43V6WNA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:41.477032712 (444h53m16.247674047s) [noise=true , address=45.91.116.6:4974 , piece_id=IKWNVS2W6ZRIFBQ4HRYMR2RJYIFBJGNGKMIQWE5WOFXOISDT2XOA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:41.690689767 (444h53m16.461331092s) [noise=true , address=45.91.116.8:4047 , piece_id=ABQ3R2LKGVF4FC2JN3ELYI7QNRKUIBAWRQ4DRLVAYKKJLHT7KRJA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:46.410141033 (444h53m21.180782368s) [noise=true , address=45.91.116.7:4010 , piece_id=BHJ2V4SGO25ECZIEDRIFEJVNAP6JK4AJ46HQA64ARTMCXVEARWGQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:46.793756941 (444h53m21.564398276s) [noise=true , address=45.91.116.13:4153 , piece_id=VAVXHRZ6D63GXY7AR6YKJDRH3HLUFSDDWQBBJCXOPCLJ6AKVJ4DQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:49.224285758 (444h53m23.994927083s) [noise=true , address=45.91.116.10:4082 , piece_id=2BW7GEGPGC5QTGNP3YWLG5ITLZ4ZAFKO4WK2UG2VMHNLAEIOJYBQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:52.677244556 (444h53m27.447885891s) [noise=true , address=77.90.31.14:4663 , piece_id=DXLYEPO3MK6SC3FNCJ7XGIXAXXSTFQ6SCKMC7I27EEZCDAOYPGVA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:54.289319789 (444h53m29.059961124s) [noise=true , address=77.90.31.12:4589 , piece_id=ZVPSEXXLSZ3ZC5IOOG26XIU2RR3LPH7QFYTA4PQC45YZ4HHYW7PQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:04:05.689973065 (444h53m40.4606144s) [noise=true , address=104.36.82.11:4115 , piece_id=6Z43WIGHE6XZYF5RVJ7H4OP5SUGJEUQB6AT7CYPJSI2WF7L2ORUA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 188.166.225.170:44524->104.36.82.11:4115: use of closed network connection; write tcp 188.166.225.170:44524->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44524->104.36.82.11:4115: use of closed network connection Jul 11 22:04:05.690022460 (444h53m40.460663805s) [noise=true , address=104.36.82.7:4909 , piece_id=3A5D2ZOUIKPV7D34I3IYAKCAZZXEVR7V4NMMGYFRFIUAIUG45EWQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 188.166.225.170:34186->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34186->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34186->104.36.82.7:4909: use of closed network connection Jul 11 22:04:05.690072436 (444h53m40.460713781s) [noise=true , address=104.36.82.2:4732 , piece_id=OZBTQI47TOFIN532E6SQFE2ES7AH2JIUI4TYVXSJQDDRSGK352AA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: write tcp 188.166.225.170:56574->104.36.82.2:4732: use of closed network connection; write tcp 188.166.225.170:56574->104.36.82.2:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56574->104.36.82.2:4732: use of closed network connection Jul 11 22:04:05.690248061 (444h53m40.460889396s) [noise=true , address=104.36.82.15:4259 , piece_id=FSTG7RKNERGMO5DQG3XY4L6WIOOAPW46FLZVENOLJ4BXU4Q3HXFQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: read tcp 188.166.225.170:43630->104.36.82.15:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:04:05.690355437 (444h53m40.460996772s) [noise=true , address=104.36.82.16:4293 , piece_id=RXEYWHR457PJV2TBV6IIFEM4JQUTBRI5XYPDFN637FJ5CSY2XLVA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 188.166.225.170:34014->104.36.82.16:4293: use of closed network connection; write tcp 188.166.225.170:34014->104.36.82.16:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34014->104.36.82.16:4293: use of closed network connection Jul 11 22:04:05.690624201 (444h53m40.461265546s) [noise=true , address=104.36.82.3:4767 , piece_id=PSBZTYYQZY7J4WQSH6NIV53URYZULANFZMFF5SQ64CZ2A4QIYVGA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 188.166.225.170:56392->104.36.82.3:4767: use of closed network connection; write tcp 188.166.225.170:56392->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56392->104.36.82.3:4767: use of closed network connection Jul 11 22:04:05.690653958 (444h53m40.461295293s) [noise=true , address=104.36.82.13:4186 , piece_id=2XLPJQOI3KB2XAY2TWJD6JGFZPL65366HKT4S7MGQEESZUSU3L7A, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:56770->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:56770->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56770->104.36.82.13:4186: use of closed network connection Jul 11 22:04:05.690728761 (444h53m40.461370096s) [noise=true , address=104.36.82.14:4221 , piece_id=3AQIO6ZO6FF5H4C4KYGIEXVLRMGGSA7MO64KQT2FXOJQL3366XCA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: write tcp 188.166.225.170:55334->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:55334->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55334->104.36.82.14:4221: use of closed network connection Jul 11 22:04:05.690758839 (444h53m40.461400164s) [seg_pos={0 0}, upload=355823]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 11 22:04:05.929272448 (444h53m40.699913773s) [upload=355823]: Done uploading segments... Jul 11 22:04:05.929299570 (444h53m40.699940905s) [upload=355823]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 21.404206303s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................X.XXX.] duration: 7.603527856s FAIL (2025-07-11 22:04:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.603583985s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 38.176231217s FAIL (2025-07-11 22:04:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 38.176242808s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 29.430750036s FAIL (2025-07-11 22:04:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682325371 (444h51m55.452966706s) [noise=true , address=77.90.31.10:4538 , piece_id=ZG3GT4YWHXI2UIRBMOQRR6OE2IB3SM2PFW3JWCOBPLXHLBX42FEA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682662677 (444h51m55.453304012s) [noise=true , address=79.127.219.38:4021 , piece_id=DXPMY6YAGV5HM7Q6SXDPGCFMDSVPLDJONZXDPHCSRGWEMFL5R5ZA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682725006 (444h51m55.453366341s) [noise=true , address=77.90.31.18:4822 , piece_id=2A5A32R2PFRIOXF4RT7SPLDA7ZYBNQRZUEF3URXGCLFIN4CPKUGQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682961879 (444h51m55.453603214s) [noise=true , address=77.90.31.13:4637 , piece_id=PIVLKJ3G3UFVE4ZPKTN2OAR4CPUZDDQF62SOJFQFWFOCIRZKGK4A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683047994 (444h51m55.453689329s) [noise=true , address=104.36.82.4:4827 , piece_id=SZSP2QHIG4QKJULROHUJ6ESM7TCP6BWS7JYCLCOYKEHC5QZ7MW2Q, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683206346 (444h51m55.453847671s) [noise=true , address=45.91.116.11:4123 , piece_id=WMPALJZGHW5TB7232YRTQ4DSJPFTTFTKMM63N2K6QIC6EDNVQYYQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683354120 (444h51m55.453995445s) [noise=true , address=104.36.82.2:4734 , piece_id=4PHLX43SMSCYJRC765X5DZDQTVKMLVZTYCFMIAQ4DRWIAYAJQ5TA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683454020 (444h51m55.454095355s) [noise=true , address=45.91.95.5:4004 , piece_id=ZWN3AVQXYBELRTX7BTDMNTHT45HXD3U5AKGXNEFJM4Q5NRSKY6YA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683651569 (444h51m55.454292904s) [noise=true , address=104.36.82.17:4357 , piece_id=Z3345C4RUT7HBFVVK6MDZLUJP4PYFNW6POEGYILDJSHADFNWKKMQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683744226 (444h51m55.454385561s) [noise=true , address=45.91.116.12:4259 , piece_id=OZMF3R53AUGH46LBKG53B7VLFGPZMMTNC5PDTVXF7U36O374WA4Q, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683956422 (444h51m55.454597757s) [noise=true , address=77.90.31.11:4588 , piece_id=L5KFV5NCKUCLJ43ABMNDDNVTVU3LOKMKH7PVS7IV6BY4PFC5BIAQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683993663 (444h51m55.454634998s) [noise=true , address=77.90.31.19:4859 , piece_id=WHLA2M2RIHPBBFJRV4CXZERFKTC2HKVRPDLGJST3LWYXGNARBS6A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684272226 (444h51m55.454913561s) [noise=true , address=45.91.116.8:4050 , piece_id=ZACWOJOY2IYEL4O6IGYLOG7KBADZ6QM26G246TSZVQ2YI6IEVBZA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684283717 (444h51m55.454925052s) [noise=true , address=77.90.31.21:4912 , piece_id=EP3FXEGB567M3G6ITDK3PQ2EP6474H33YPGJP5QAM2FVXOF5GP2A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684307864 (444h51m55.454949199s) [noise=true , address=79.127.201.215:4350 , piece_id=QXT4NW3N627QSQPXW3WQUOA6FHK4AZ3JQISSZBZEWD7TJXVCG3YQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684520521 (444h51m55.455161846s) [noise=true , address=104.36.82.38:4692 , piece_id=PFGTVLSRSDMWTMOWGCYVFOUHSD3ERCW5PH4HP3ZMPUPEO34MA5UQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684710494 (444h51m55.455351829s) [noise=true , address=104.36.82.21:4490 , piece_id=TRIYXTVLJDLJLCGO64BUAJ2AXKNQPZIQNJHI2HBZGYYWQICYXEUA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:22.030909036 (444h51m56.801550361s) [upload=355807]: Next returned nil segment Jul 11 22:02:22.030926990 (444h51m56.801568325s) [upload=355807]: Waiting for error group managing segments... Jul 11 22:02:22.574284084 (444h51m57.344925409s) [noise=true , address=79.127.201.215:4350 , piece_id=QXT4NW3N627QSQPXW3WQUOA6FHK4AZ3JQISSZBZEWD7TJXVCG3YQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.574330413 (444h51m57.344971748s) [noise=true , address=45.91.95.5:4004 , piece_id=ZWN3AVQXYBELRTX7BTDMNTHT45HXD3U5AKGXNEFJM4Q5NRSKY6YA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.606134009 (444h51m57.376775344s) [noise=true , address=45.91.116.11:4123 , piece_id=WMPALJZGHW5TB7232YRTQ4DSJPFTTFTKMM63N2K6QIC6EDNVQYYQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.607118172 (444h51m57.377759507s) [noise=true , address=45.91.116.12:4259 , piece_id=OZMF3R53AUGH46LBKG53B7VLFGPZMMTNC5PDTVXF7U36O374WA4Q, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.618767719 (444h51m57.389409054s) [noise=true , address=77.90.31.18:4822 , piece_id=2A5A32R2PFRIOXF4RT7SPLDA7ZYBNQRZUEF3URXGCLFIN4CPKUGQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.623246287 (444h51m57.393887622s) [noise=true , address=77.90.31.15:4709 , piece_id=MNYBVFYI3APDTCXKNQ27PJB3NEOXK5NNJSTGO3L2KIGG7CQ6QHZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.625221406 (444h51m57.395862742s) [noise=true , address=45.91.116.8:4050 , piece_id=ZACWOJOY2IYEL4O6IGYLOG7KBADZ6QM26G246TSZVQ2YI6IEVBZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.650485021 (444h51m57.421126356s) [noise=true , address=77.90.31.11:4588 , piece_id=L5KFV5NCKUCLJ43ABMNDDNVTVU3LOKMKH7PVS7IV6BY4PFC5BIAQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.658800550 (444h51m57.429441885s) [noise=true , address=77.90.31.13:4637 , piece_id=PIVLKJ3G3UFVE4ZPKTN2OAR4CPUZDDQF62SOJFQFWFOCIRZKGK4A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.660254201 (444h51m57.430895526s) [noise=true , address=77.90.31.10:4538 , piece_id=ZG3GT4YWHXI2UIRBMOQRR6OE2IB3SM2PFW3JWCOBPLXHLBX42FEA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.663604691 (444h51m57.434246026s) [noise=true , address=77.90.31.19:4859 , piece_id=WHLA2M2RIHPBBFJRV4CXZERFKTC2HKVRPDLGJST3LWYXGNARBS6A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.667276506 (444h51m57.437917841s) [noise=true , address=77.90.31.21:4912 , piece_id=EP3FXEGB567M3G6ITDK3PQ2EP6474H33YPGJP5QAM2FVXOF5GP2A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.777412707 (444h51m57.548054042s) [noise=true , address=79.127.219.38:4021 , piece_id=DXPMY6YAGV5HM7Q6SXDPGCFMDSVPLDJONZXDPHCSRGWEMFL5R5ZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:03:19.601895201 (444h52m54.372536526s) [noise=true , address=104.36.82.4:4827 , piece_id=SZSP2QHIG4QKJULROHUJ6ESM7TCP6BWS7JYCLCOYKEHC5QZ7MW2Q, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1VXFTn1urnJ2qkUcDc5785hJ8L6bNVQTtgcgNeWLaZ9wEkKYVx, address:104.36.82.4:4827): protocol: write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection; write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection Jul 11 22:03:19.601925429 (444h52m54.372566764s) [noise=true , address=104.36.82.21:4490 , piece_id=TRIYXTVLJDLJLCGO64BUAJ2AXKNQPZIQNJHI2HBZGYYWQICYXEUA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:12DPUW1Ej5FLg9yTbhqdWBX8YAryWG82BnVegsCiTRX71pUA24n, address:104.36.82.21:4490): protocol: write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection; write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection Jul 11 22:03:19.601964724 (444h52m54.372606049s) [noise=true , address=104.36.82.2:4734 , piece_id=4PHLX43SMSCYJRC765X5DZDQTVKMLVZTYCFMIAQ4DRWIAYAJQ5TA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:12GLb5LQje9wwivnj8Wj1tXcnS6mFb2Bco5saNwgmFJBVDujrv7, address:104.36.82.2:4734): protocol: write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection; write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection Jul 11 22:03:19.601995563 (444h52m54.372636898s) [noise=true , address=104.36.82.38:4692 , piece_id=PFGTVLSRSDMWTMOWGCYVFOUHSD3ERCW5PH4HP3ZMPUPEO34MA5UQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:104.36.82.38:4692): protocol: write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection; write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection Jul 11 22:03:19.602044546 (444h52m54.372685881s) [noise=true , address=104.36.82.17:4357 , piece_id=Z3345C4RUT7HBFVVK6MDZLUJP4PYFNW6POEGYILDJSHADFNWKKMQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:104.36.82.17:4357): protocol: read tcp 188.166.225.170:60360->104.36.82.17:4357: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:03:19.602071177 (444h52m54.372712502s) [seg_pos={0 0}, upload=355807]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:03:19.829114955 (444h52m54.59975629s) [upload=355807]: Done uploading segments... Jul 11 22:03:19.829139813 (444h52m54.599781148s) [upload=355807]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 29.430791204s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.240761773s FAIL (2025-07-11 22:04:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 63 , piece_id=DXLYEPO3MK6SC3FNCJ7XGIXAXXSTFQ6SCKMC7I27EEZCDAOYPGVA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.588324207 (444h53m11.358965542s) [noise=true , address=45.91.116.10:4082 , piece_id=2BW7GEGPGC5QTGNP3YWLG5ITLZ4ZAFKO4WK2UG2VMHNLAEIOJYBQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.588630102 (444h53m11.359271457s) [noise=true , address=45.91.116.7:4010 , piece_id=BHJ2V4SGO25ECZIEDRIFEJVNAP6JK4AJ46HQA64ARTMCXVEARWGQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.588716027 (444h53m11.359357362s) [noise=true , address=104.36.82.14:4221 , piece_id=3AQIO6ZO6FF5H4C4KYGIEXVLRMGGSA7MO64KQT2FXOJQL3366XCA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.588939384 (444h53m11.359580719s) [noise=true , address=104.36.82.16:4293 , piece_id=RXEYWHR457PJV2TBV6IIFEM4JQUTBRI5XYPDFN637FJ5CSY2XLVA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589064544 (444h53m11.359705869s) [noise=true , address=45.91.116.8:4047 , piece_id=ABQ3R2LKGVF4FC2JN3ELYI7QNRKUIBAWRQ4DRLVAYKKJLHT7KRJA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589149476 (444h53m11.359790811s) [noise=true , address=104.36.82.2:4732 , piece_id=OZBTQI47TOFIN532E6SQFE2ES7AH2JIUI4TYVXSJQDDRSGK352AA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589406508 (444h53m11.360047843s) [noise=true , address=104.36.82.11:4115 , piece_id=6Z43WIGHE6XZYF5RVJ7H4OP5SUGJEUQB6AT7CYPJSI2WF7L2ORUA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589548579 (444h53m11.360189914s) [noise=true , address=104.36.82.15:4259 , piece_id=FSTG7RKNERGMO5DQG3XY4L6WIOOAPW46FLZVENOLJ4BXU4Q3HXFQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589679801 (444h53m11.360321136s) [noise=true , address=45.91.116.13:4153 , piece_id=VAVXHRZ6D63GXY7AR6YKJDRH3HLUFSDDWQBBJCXOPCLJ6AKVJ4DQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.589939458 (444h53m11.360580793s) [noise=true , address=104.36.82.7:4909 , piece_id=3A5D2ZOUIKPV7D34I3IYAKCAZZXEVR7V4NMMGYFRFIUAIUG45EWQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.590036683 (444h53m11.360678019s) [noise=true , address=45.91.116.11:4119 , piece_id=RNPSA3BWL4BVYCH6WIPK54WZMP5IZ35LNZQRGQCZ5FLXH43V6WNA, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.594031947 (444h53m11.364673282s) [noise=true , address=104.36.82.13:4186 , piece_id=2XLPJQOI3KB2XAY2TWJD6JGFZPL65366HKT4S7MGQEESZUSU3L7A, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:36.594404540 (444h53m11.365045875s) [noise=true , address=77.90.31.12:4589 , piece_id=ZVPSEXXLSZ3ZC5IOOG26XIU2RR3LPH7QFYTA4PQC45YZ4HHYW7PQ, seg_pos={0 0}, upload=355823]: Uploading piece... Jul 11 22:03:39.469032713 (444h53m14.239674038s) [upload=355823]: Next returned nil segment Jul 11 22:03:39.469049616 (444h53m14.239690941s) [upload=355823]: Waiting for error group managing segments... Jul 11 22:03:40.688983051 (444h53m15.459624386s) [noise=true , address=45.91.116.11:4119 , piece_id=RNPSA3BWL4BVYCH6WIPK54WZMP5IZ35LNZQRGQCZ5FLXH43V6WNA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:41.477032712 (444h53m16.247674047s) [noise=true , address=45.91.116.6:4974 , piece_id=IKWNVS2W6ZRIFBQ4HRYMR2RJYIFBJGNGKMIQWE5WOFXOISDT2XOA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:41.690689767 (444h53m16.461331092s) [noise=true , address=45.91.116.8:4047 , piece_id=ABQ3R2LKGVF4FC2JN3ELYI7QNRKUIBAWRQ4DRLVAYKKJLHT7KRJA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:46.410141033 (444h53m21.180782368s) [noise=true , address=45.91.116.7:4010 , piece_id=BHJ2V4SGO25ECZIEDRIFEJVNAP6JK4AJ46HQA64ARTMCXVEARWGQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:46.793756941 (444h53m21.564398276s) [noise=true , address=45.91.116.13:4153 , piece_id=VAVXHRZ6D63GXY7AR6YKJDRH3HLUFSDDWQBBJCXOPCLJ6AKVJ4DQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:49.224285758 (444h53m23.994927083s) [noise=true , address=45.91.116.10:4082 , piece_id=2BW7GEGPGC5QTGNP3YWLG5ITLZ4ZAFKO4WK2UG2VMHNLAEIOJYBQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:52.677244556 (444h53m27.447885891s) [noise=true , address=77.90.31.14:4663 , piece_id=DXLYEPO3MK6SC3FNCJ7XGIXAXXSTFQ6SCKMC7I27EEZCDAOYPGVA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:03:54.289319789 (444h53m29.059961124s) [noise=true , address=77.90.31.12:4589 , piece_id=ZVPSEXXLSZ3ZC5IOOG26XIU2RR3LPH7QFYTA4PQC45YZ4HHYW7PQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: <nil> Jul 11 22:04:05.689973065 (444h53m40.4606144s) [noise=true , address=104.36.82.11:4115 , piece_id=6Z43WIGHE6XZYF5RVJ7H4OP5SUGJEUQB6AT7CYPJSI2WF7L2ORUA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 188.166.225.170:44524->104.36.82.11:4115: use of closed network connection; write tcp 188.166.225.170:44524->104.36.82.11:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44524->104.36.82.11:4115: use of closed network connection Jul 11 22:04:05.690022460 (444h53m40.460663805s) [noise=true , address=104.36.82.7:4909 , piece_id=3A5D2ZOUIKPV7D34I3IYAKCAZZXEVR7V4NMMGYFRFIUAIUG45EWQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 188.166.225.170:34186->104.36.82.7:4909: use of closed network connection; write tcp 188.166.225.170:34186->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34186->104.36.82.7:4909: use of closed network connection Jul 11 22:04:05.690072436 (444h53m40.460713781s) [noise=true , address=104.36.82.2:4732 , piece_id=OZBTQI47TOFIN532E6SQFE2ES7AH2JIUI4TYVXSJQDDRSGK352AA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: write tcp 188.166.225.170:56574->104.36.82.2:4732: use of closed network connection; write tcp 188.166.225.170:56574->104.36.82.2:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56574->104.36.82.2:4732: use of closed network connection Jul 11 22:04:05.690248061 (444h53m40.460889396s) [noise=true , address=104.36.82.15:4259 , piece_id=FSTG7RKNERGMO5DQG3XY4L6WIOOAPW46FLZVENOLJ4BXU4Q3HXFQ, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: read tcp 188.166.225.170:43630->104.36.82.15:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:04:05.690355437 (444h53m40.460996772s) [noise=true , address=104.36.82.16:4293 , piece_id=RXEYWHR457PJV2TBV6IIFEM4JQUTBRI5XYPDFN637FJ5CSY2XLVA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 188.166.225.170:34014->104.36.82.16:4293: use of closed network connection; write tcp 188.166.225.170:34014->104.36.82.16:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34014->104.36.82.16:4293: use of closed network connection Jul 11 22:04:05.690624201 (444h53m40.461265546s) [noise=true , address=104.36.82.3:4767 , piece_id=PSBZTYYQZY7J4WQSH6NIV53URYZULANFZMFF5SQ64CZ2A4QIYVGA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 188.166.225.170:56392->104.36.82.3:4767: use of closed network connection; write tcp 188.166.225.170:56392->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56392->104.36.82.3:4767: use of closed network connection Jul 11 22:04:05.690653958 (444h53m40.461295293s) [noise=true , address=104.36.82.13:4186 , piece_id=2XLPJQOI3KB2XAY2TWJD6JGFZPL65366HKT4S7MGQEESZUSU3L7A, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:56770->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:56770->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56770->104.36.82.13:4186: use of closed network connection Jul 11 22:04:05.690728761 (444h53m40.461370096s) [noise=true , address=104.36.82.14:4221 , piece_id=3AQIO6ZO6FF5H4C4KYGIEXVLRMGGSA7MO64KQT2FXOJQL3366XCA, seg_pos={0 0}, upload=355823]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: write tcp 188.166.225.170:55334->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:55334->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55334->104.36.82.14:4221: use of closed network connection Jul 11 22:04:05.690758839 (444h53m40.461400164s) [seg_pos={0 0}, upload=355823]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 11 22:04:05.929272448 (444h53m40.699913773s) [upload=355823]: Done uploading segments... Jul 11 22:04:05.929299570 (444h53m40.699940905s) [upload=355823]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.240785497s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [X.........................................................XX] duration: 1m0.001432011s FAIL (2025-07-11 22:03:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00144217s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X......................................................X.XXX] duration: 30.000463147s FAIL (2025-07-11 22:03:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000483096s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 16.408513065s FAIL (2025-07-11 22:03:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682325371 (444h51m55.452966706s) [noise=true , address=77.90.31.10:4538 , piece_id=ZG3GT4YWHXI2UIRBMOQRR6OE2IB3SM2PFW3JWCOBPLXHLBX42FEA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682662677 (444h51m55.453304012s) [noise=true , address=79.127.219.38:4021 , piece_id=DXPMY6YAGV5HM7Q6SXDPGCFMDSVPLDJONZXDPHCSRGWEMFL5R5ZA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682725006 (444h51m55.453366341s) [noise=true , address=77.90.31.18:4822 , piece_id=2A5A32R2PFRIOXF4RT7SPLDA7ZYBNQRZUEF3URXGCLFIN4CPKUGQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682961879 (444h51m55.453603214s) [noise=true , address=77.90.31.13:4637 , piece_id=PIVLKJ3G3UFVE4ZPKTN2OAR4CPUZDDQF62SOJFQFWFOCIRZKGK4A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683047994 (444h51m55.453689329s) [noise=true , address=104.36.82.4:4827 , piece_id=SZSP2QHIG4QKJULROHUJ6ESM7TCP6BWS7JYCLCOYKEHC5QZ7MW2Q, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683206346 (444h51m55.453847671s) [noise=true , address=45.91.116.11:4123 , piece_id=WMPALJZGHW5TB7232YRTQ4DSJPFTTFTKMM63N2K6QIC6EDNVQYYQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683354120 (444h51m55.453995445s) [noise=true , address=104.36.82.2:4734 , piece_id=4PHLX43SMSCYJRC765X5DZDQTVKMLVZTYCFMIAQ4DRWIAYAJQ5TA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683454020 (444h51m55.454095355s) [noise=true , address=45.91.95.5:4004 , piece_id=ZWN3AVQXYBELRTX7BTDMNTHT45HXD3U5AKGXNEFJM4Q5NRSKY6YA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683651569 (444h51m55.454292904s) [noise=true , address=104.36.82.17:4357 , piece_id=Z3345C4RUT7HBFVVK6MDZLUJP4PYFNW6POEGYILDJSHADFNWKKMQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683744226 (444h51m55.454385561s) [noise=true , address=45.91.116.12:4259 , piece_id=OZMF3R53AUGH46LBKG53B7VLFGPZMMTNC5PDTVXF7U36O374WA4Q, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683956422 (444h51m55.454597757s) [noise=true , address=77.90.31.11:4588 , piece_id=L5KFV5NCKUCLJ43ABMNDDNVTVU3LOKMKH7PVS7IV6BY4PFC5BIAQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683993663 (444h51m55.454634998s) [noise=true , address=77.90.31.19:4859 , piece_id=WHLA2M2RIHPBBFJRV4CXZERFKTC2HKVRPDLGJST3LWYXGNARBS6A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684272226 (444h51m55.454913561s) [noise=true , address=45.91.116.8:4050 , piece_id=ZACWOJOY2IYEL4O6IGYLOG7KBADZ6QM26G246TSZVQ2YI6IEVBZA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684283717 (444h51m55.454925052s) [noise=true , address=77.90.31.21:4912 , piece_id=EP3FXEGB567M3G6ITDK3PQ2EP6474H33YPGJP5QAM2FVXOF5GP2A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684307864 (444h51m55.454949199s) [noise=true , address=79.127.201.215:4350 , piece_id=QXT4NW3N627QSQPXW3WQUOA6FHK4AZ3JQISSZBZEWD7TJXVCG3YQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684520521 (444h51m55.455161846s) [noise=true , address=104.36.82.38:4692 , piece_id=PFGTVLSRSDMWTMOWGCYVFOUHSD3ERCW5PH4HP3ZMPUPEO34MA5UQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684710494 (444h51m55.455351829s) [noise=true , address=104.36.82.21:4490 , piece_id=TRIYXTVLJDLJLCGO64BUAJ2AXKNQPZIQNJHI2HBZGYYWQICYXEUA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:22.030909036 (444h51m56.801550361s) [upload=355807]: Next returned nil segment Jul 11 22:02:22.030926990 (444h51m56.801568325s) [upload=355807]: Waiting for error group managing segments... Jul 11 22:02:22.574284084 (444h51m57.344925409s) [noise=true , address=79.127.201.215:4350 , piece_id=QXT4NW3N627QSQPXW3WQUOA6FHK4AZ3JQISSZBZEWD7TJXVCG3YQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.574330413 (444h51m57.344971748s) [noise=true , address=45.91.95.5:4004 , piece_id=ZWN3AVQXYBELRTX7BTDMNTHT45HXD3U5AKGXNEFJM4Q5NRSKY6YA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.606134009 (444h51m57.376775344s) [noise=true , address=45.91.116.11:4123 , piece_id=WMPALJZGHW5TB7232YRTQ4DSJPFTTFTKMM63N2K6QIC6EDNVQYYQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.607118172 (444h51m57.377759507s) [noise=true , address=45.91.116.12:4259 , piece_id=OZMF3R53AUGH46LBKG53B7VLFGPZMMTNC5PDTVXF7U36O374WA4Q, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.618767719 (444h51m57.389409054s) [noise=true , address=77.90.31.18:4822 , piece_id=2A5A32R2PFRIOXF4RT7SPLDA7ZYBNQRZUEF3URXGCLFIN4CPKUGQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.623246287 (444h51m57.393887622s) [noise=true , address=77.90.31.15:4709 , piece_id=MNYBVFYI3APDTCXKNQ27PJB3NEOXK5NNJSTGO3L2KIGG7CQ6QHZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.625221406 (444h51m57.395862742s) [noise=true , address=45.91.116.8:4050 , piece_id=ZACWOJOY2IYEL4O6IGYLOG7KBADZ6QM26G246TSZVQ2YI6IEVBZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.650485021 (444h51m57.421126356s) [noise=true , address=77.90.31.11:4588 , piece_id=L5KFV5NCKUCLJ43ABMNDDNVTVU3LOKMKH7PVS7IV6BY4PFC5BIAQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.658800550 (444h51m57.429441885s) [noise=true , address=77.90.31.13:4637 , piece_id=PIVLKJ3G3UFVE4ZPKTN2OAR4CPUZDDQF62SOJFQFWFOCIRZKGK4A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.660254201 (444h51m57.430895526s) [noise=true , address=77.90.31.10:4538 , piece_id=ZG3GT4YWHXI2UIRBMOQRR6OE2IB3SM2PFW3JWCOBPLXHLBX42FEA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.663604691 (444h51m57.434246026s) [noise=true , address=77.90.31.19:4859 , piece_id=WHLA2M2RIHPBBFJRV4CXZERFKTC2HKVRPDLGJST3LWYXGNARBS6A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.667276506 (444h51m57.437917841s) [noise=true , address=77.90.31.21:4912 , piece_id=EP3FXEGB567M3G6ITDK3PQ2EP6474H33YPGJP5QAM2FVXOF5GP2A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.777412707 (444h51m57.548054042s) [noise=true , address=79.127.219.38:4021 , piece_id=DXPMY6YAGV5HM7Q6SXDPGCFMDSVPLDJONZXDPHCSRGWEMFL5R5ZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:03:19.601895201 (444h52m54.372536526s) [noise=true , address=104.36.82.4:4827 , piece_id=SZSP2QHIG4QKJULROHUJ6ESM7TCP6BWS7JYCLCOYKEHC5QZ7MW2Q, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1VXFTn1urnJ2qkUcDc5785hJ8L6bNVQTtgcgNeWLaZ9wEkKYVx, address:104.36.82.4:4827): protocol: write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection; write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection Jul 11 22:03:19.601925429 (444h52m54.372566764s) [noise=true , address=104.36.82.21:4490 , piece_id=TRIYXTVLJDLJLCGO64BUAJ2AXKNQPZIQNJHI2HBZGYYWQICYXEUA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:12DPUW1Ej5FLg9yTbhqdWBX8YAryWG82BnVegsCiTRX71pUA24n, address:104.36.82.21:4490): protocol: write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection; write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection Jul 11 22:03:19.601964724 (444h52m54.372606049s) [noise=true , address=104.36.82.2:4734 , piece_id=4PHLX43SMSCYJRC765X5DZDQTVKMLVZTYCFMIAQ4DRWIAYAJQ5TA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:12GLb5LQje9wwivnj8Wj1tXcnS6mFb2Bco5saNwgmFJBVDujrv7, address:104.36.82.2:4734): protocol: write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection; write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection Jul 11 22:03:19.601995563 (444h52m54.372636898s) [noise=true , address=104.36.82.38:4692 , piece_id=PFGTVLSRSDMWTMOWGCYVFOUHSD3ERCW5PH4HP3ZMPUPEO34MA5UQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:104.36.82.38:4692): protocol: write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection; write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection Jul 11 22:03:19.602044546 (444h52m54.372685881s) [noise=true , address=104.36.82.17:4357 , piece_id=Z3345C4RUT7HBFVVK6MDZLUJP4PYFNW6POEGYILDJSHADFNWKKMQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:104.36.82.17:4357): protocol: read tcp 188.166.225.170:60360->104.36.82.17:4357: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:03:19.602071177 (444h52m54.372712502s) [seg_pos={0 0}, upload=355807]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:03:19.829114955 (444h52m54.59975629s) [upload=355807]: Done uploading segments... Jul 11 22:03:19.829139813 (444h52m54.599781148s) [upload=355807]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 16.408540377s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.237186673s FAIL (2025-07-11 22:03:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Jul 11 22:02:55.084268240 (444h52m29.854909575s) [noise=true , address=45.91.116.4:4938 , piece_id=NM2VJAEPMCJYD3ECQ5YBPBXJK5YX6GGQ4EJVDNJY3QPQBKEZZCIQ, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.084308757 (444h52m29.854950092s) [noise=true , address=104.36.82.9:4006 , piece_id=YNT6MYJWGR32XUBALINK4ID6PZN4G4NAM6E7XGI7O4TSTF3XY3JA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.084698352 (444h52m29.855339687s) [noise=true , address=45.91.116.11:4118 , piece_id=EU5ISFTVFLJ5VDRHH4I7643MWPVAC6NDMDYQ6N56HDY7Z36Y2FVA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.084867556 (444h52m29.855508891s) [noise=true , address=104.36.82.6:4873 , piece_id=KIEKHASUS3SJFI6G62FODAHNX6UM4DTYIXJ2SUWOYAWBKVZDJF6A, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.084970002 (444h52m29.855611327s) [noise=true , address=77.90.31.14:4662 , piece_id=T3XG6RPPI3WQMEMOECT74IIW3EXX572MOJRMLVDRLSIKQOZFKZKQ, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.085229127 (444h52m29.855870463s) [noise=true , address=45.91.116.9:4263 , piece_id=ZJP7SMVKC7H4ZUWMMYRDE7XY32F5KGJSHZWVUYFRD26DJ6TZKAQA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.085339599 (444h52m29.855980934s) [noise=true , address=45.91.116.8:4048 , piece_id=E5Y424MTDHONFYEODCYIXXHRBD7HX47WHOHHUEJ5MJQUGKVUE37A, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.085493263 (444h52m29.856134588s) [noise=true , address=104.36.82.16:4295 , piece_id=DIGYZTOSDYZFE23366OO2RRCYDW5KBYXUKCB5BZHYBH7ZNACL5SQ, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.085779781 (444h52m29.856421116s) [noise=true , address=45.91.116.12:4225 , piece_id=KNCNBAWQPUU23ATIFALBUSYNV4T3L2CTPT4NDISAQ5WDXL3ZILXQ, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.086651209 (444h52m29.857292544s) [noise=true , address=104.36.82.7:4912 , piece_id=BRCUBQYBSNRTQ2OTHQCMX6KT4VFAYHEVRDDQRBB7F4BZERYXUBOA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.087624651 (444h52m29.858265986s) [noise=true , address=77.90.31.18:4805 , piece_id=ZBDXGLBISTN74DREVDLATH3I24HC4KVKF4QF4ISAXJ6LTP3PZTFA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.087899256 (444h52m29.858540591s) [noise=true , address=104.36.82.12:4151 , piece_id=XNSKGRQSFS6SVN6MA6XHOUYXOGHEOSZBMM57NW3MAJKOWUJI3JUA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.088186626 (444h52m29.858827962s) [noise=true , address=77.90.31.12:4589 , piece_id=GAR7DWQIXLMHRABNGGI7GKHX2TNG4IWOMFF4NY5EPFFCJEZAUKLA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.088812213 (444h52m29.859453548s) [noise=true , address=104.36.82.15:4258 , piece_id=AMHITBZYNTDELDI2KCCHB3OZC7AGE7EBOTI2RXEIX2EBGB27D5WQ, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.089366563 (444h52m29.860007898s) [noise=true , address=45.91.116.5:4299 , piece_id=AAOWP7XVLD6UHDXN42YVGS6IX6SHRR2DYZYHRH6U4VRVXC444WYA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:55.089671657 (444h52m29.860312992s) [noise=true , address=104.36.82.14:4222 , piece_id=VTCD7ZW3FJJRCADQPQKZVLDH2NJCMBM2UVUWQ6SMJSPAT3F4AILA, seg_pos={0 0}, upload=355814]: Uploading piece... Jul 11 22:02:58.729301111 (444h52m33.499942456s) [upload=355814]: Next returned nil segment Jul 11 22:02:58.729322051 (444h52m33.499963416s) [upload=355814]: Waiting for error group managing segments... Jul 11 22:03:00.363787782 (444h52m35.134429117s) [noise=true , address=77.90.31.18:4805 , piece_id=ZBDXGLBISTN74DREVDLATH3I24HC4KVKF4QF4ISAXJ6LTP3PZTFA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:10.458531718 (444h52m45.229173043s) [noise=true , address=45.91.116.11:4118 , piece_id=EU5ISFTVFLJ5VDRHH4I7643MWPVAC6NDMDYQ6N56HDY7Z36Y2FVA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:10.465119722 (444h52m45.235761057s) [noise=true , address=45.91.116.12:4225 , piece_id=KNCNBAWQPUU23ATIFALBUSYNV4T3L2CTPT4NDISAQ5WDXL3ZILXQ, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:11.011051984 (444h52m45.781693359s) [noise=true , address=45.91.116.5:4299 , piece_id=AAOWP7XVLD6UHDXN42YVGS6IX6SHRR2DYZYHRH6U4VRVXC444WYA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:12.274846200 (444h52m47.045487535s) [noise=true , address=45.91.116.9:4263 , piece_id=ZJP7SMVKC7H4ZUWMMYRDE7XY32F5KGJSHZWVUYFRD26DJ6TZKAQA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:13.040686315 (444h52m47.81132765s) [noise=true , address=45.91.116.8:4048 , piece_id=E5Y424MTDHONFYEODCYIXXHRBD7HX47WHOHHUEJ5MJQUGKVUE37A, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:13.667500680 (444h52m48.438142005s) [noise=true , address=77.90.31.14:4662 , piece_id=T3XG6RPPI3WQMEMOECT74IIW3EXX572MOJRMLVDRLSIKQOZFKZKQ, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:14.319566222 (444h52m49.090207557s) [noise=true , address=45.91.116.4:4938 , piece_id=NM2VJAEPMCJYD3ECQ5YBPBXJK5YX6GGQ4EJVDNJY3QPQBKEZZCIQ, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:21.998474016 (444h52m56.769115351s) [noise=true , address=77.90.31.12:4589 , piece_id=GAR7DWQIXLMHRABNGGI7GKHX2TNG4IWOMFF4NY5EPFFCJEZAUKLA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: <nil> Jul 11 22:03:24.219681212 (444h52m58.990322547s) [noise=true , address=104.36.82.6:4873 , piece_id=KIEKHASUS3SJFI6G62FODAHNX6UM4DTYIXJ2SUWOYAWBKVZDJF6A, seg_pos={0 0}, upload=355814]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: write tcp 188.166.225.170:46522->104.36.82.6:4873: use of closed network connection; write tcp 188.166.225.170:46522->104.36.82.6:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46522->104.36.82.6:4873: use of closed network connection Jul 11 22:03:24.219693156 (444h52m58.990334491s) [noise=true , address=104.36.82.16:4295 , piece_id=DIGYZTOSDYZFE23366OO2RRCYDW5KBYXUKCB5BZHYBH7ZNACL5SQ, seg_pos={0 0}, upload=355814]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 188.166.225.170:48582->104.36.82.16:4295: use of closed network connection; write tcp 188.166.225.170:48582->104.36.82.16:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48582->104.36.82.16:4295: use of closed network connection Jul 11 22:03:24.219736017 (444h52m58.990377352s) [noise=true , address=104.36.82.14:4222 , piece_id=VTCD7ZW3FJJRCADQPQKZVLDH2NJCMBM2UVUWQ6SMJSPAT3F4AILA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 188.166.225.170:39256->104.36.82.14:4222: use of closed network connection; write tcp 188.166.225.170:39256->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39256->104.36.82.14:4222: use of closed network connection Jul 11 22:03:24.219745155 (444h52m58.99038649s) [noise=true , address=104.36.82.15:4258 , piece_id=AMHITBZYNTDELDI2KCCHB3OZC7AGE7EBOTI2RXEIX2EBGB27D5WQ, seg_pos={0 0}, upload=355814]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 188.166.225.170:42780->104.36.82.15:4258: use of closed network connection; write tcp 188.166.225.170:42780->104.36.82.15:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42780->104.36.82.15:4258: use of closed network connection Jul 11 22:03:24.219776134 (444h52m58.990417469s) [noise=true , address=104.36.82.7:4912 , piece_id=BRCUBQYBSNRTQ2OTHQCMX6KT4VFAYHEVRDDQRBB7F4BZERYXUBOA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: write tcp 188.166.225.170:40190->104.36.82.7:4912: use of closed network connection; write tcp 188.166.225.170:40190->104.36.82.7:4912: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40190->104.36.82.7:4912: use of closed network connection Jul 11 22:03:24.219788367 (444h52m58.990429702s) [noise=true , address=104.36.82.9:4006 , piece_id=YNT6MYJWGR32XUBALINK4ID6PZN4G4NAM6E7XGI7O4TSTF3XY3JA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 188.166.225.170:54626->104.36.82.9:4006: use of closed network connection; write tcp 188.166.225.170:54626->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54626->104.36.82.9:4006: use of closed network connection Jul 11 22:03:24.219868902 (444h52m58.990510237s) [noise=true , address=104.36.82.12:4151 , piece_id=XNSKGRQSFS6SVN6MA6XHOUYXOGHEOSZBMM57NW3MAJKOWUJI3JUA, seg_pos={0 0}, upload=355814]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: context deadline exceeded; context deadline exceeded Jul 11 22:03:24.219884041 (444h52m58.990525366s) [seg_pos={0 0}, upload=355814]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 11 22:03:24.455823527 (444h52m59.226464862s) [upload=355814]: Done uploading segments... Jul 11 22:03:24.455846922 (444h52m59.226488257s) [upload=355814]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.237248311s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.229352169s FAIL (2025-07-11 22:03:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682325371 (444h51m55.452966706s) [noise=true , address=77.90.31.10:4538 , piece_id=ZG3GT4YWHXI2UIRBMOQRR6OE2IB3SM2PFW3JWCOBPLXHLBX42FEA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682662677 (444h51m55.453304012s) [noise=true , address=79.127.219.38:4021 , piece_id=DXPMY6YAGV5HM7Q6SXDPGCFMDSVPLDJONZXDPHCSRGWEMFL5R5ZA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682725006 (444h51m55.453366341s) [noise=true , address=77.90.31.18:4822 , piece_id=2A5A32R2PFRIOXF4RT7SPLDA7ZYBNQRZUEF3URXGCLFIN4CPKUGQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.682961879 (444h51m55.453603214s) [noise=true , address=77.90.31.13:4637 , piece_id=PIVLKJ3G3UFVE4ZPKTN2OAR4CPUZDDQF62SOJFQFWFOCIRZKGK4A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683047994 (444h51m55.453689329s) [noise=true , address=104.36.82.4:4827 , piece_id=SZSP2QHIG4QKJULROHUJ6ESM7TCP6BWS7JYCLCOYKEHC5QZ7MW2Q, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683206346 (444h51m55.453847671s) [noise=true , address=45.91.116.11:4123 , piece_id=WMPALJZGHW5TB7232YRTQ4DSJPFTTFTKMM63N2K6QIC6EDNVQYYQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683354120 (444h51m55.453995445s) [noise=true , address=104.36.82.2:4734 , piece_id=4PHLX43SMSCYJRC765X5DZDQTVKMLVZTYCFMIAQ4DRWIAYAJQ5TA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683454020 (444h51m55.454095355s) [noise=true , address=45.91.95.5:4004 , piece_id=ZWN3AVQXYBELRTX7BTDMNTHT45HXD3U5AKGXNEFJM4Q5NRSKY6YA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683651569 (444h51m55.454292904s) [noise=true , address=104.36.82.17:4357 , piece_id=Z3345C4RUT7HBFVVK6MDZLUJP4PYFNW6POEGYILDJSHADFNWKKMQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683744226 (444h51m55.454385561s) [noise=true , address=45.91.116.12:4259 , piece_id=OZMF3R53AUGH46LBKG53B7VLFGPZMMTNC5PDTVXF7U36O374WA4Q, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683956422 (444h51m55.454597757s) [noise=true , address=77.90.31.11:4588 , piece_id=L5KFV5NCKUCLJ43ABMNDDNVTVU3LOKMKH7PVS7IV6BY4PFC5BIAQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.683993663 (444h51m55.454634998s) [noise=true , address=77.90.31.19:4859 , piece_id=WHLA2M2RIHPBBFJRV4CXZERFKTC2HKVRPDLGJST3LWYXGNARBS6A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684272226 (444h51m55.454913561s) [noise=true , address=45.91.116.8:4050 , piece_id=ZACWOJOY2IYEL4O6IGYLOG7KBADZ6QM26G246TSZVQ2YI6IEVBZA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684283717 (444h51m55.454925052s) [noise=true , address=77.90.31.21:4912 , piece_id=EP3FXEGB567M3G6ITDK3PQ2EP6474H33YPGJP5QAM2FVXOF5GP2A, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684307864 (444h51m55.454949199s) [noise=true , address=79.127.201.215:4350 , piece_id=QXT4NW3N627QSQPXW3WQUOA6FHK4AZ3JQISSZBZEWD7TJXVCG3YQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684520521 (444h51m55.455161846s) [noise=true , address=104.36.82.38:4692 , piece_id=PFGTVLSRSDMWTMOWGCYVFOUHSD3ERCW5PH4HP3ZMPUPEO34MA5UQ, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:20.684710494 (444h51m55.455351829s) [noise=true , address=104.36.82.21:4490 , piece_id=TRIYXTVLJDLJLCGO64BUAJ2AXKNQPZIQNJHI2HBZGYYWQICYXEUA, seg_pos={0 0}, upload=355807]: Uploading piece... Jul 11 22:02:22.030909036 (444h51m56.801550361s) [upload=355807]: Next returned nil segment Jul 11 22:02:22.030926990 (444h51m56.801568325s) [upload=355807]: Waiting for error group managing segments... Jul 11 22:02:22.574284084 (444h51m57.344925409s) [noise=true , address=79.127.201.215:4350 , piece_id=QXT4NW3N627QSQPXW3WQUOA6FHK4AZ3JQISSZBZEWD7TJXVCG3YQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.574330413 (444h51m57.344971748s) [noise=true , address=45.91.95.5:4004 , piece_id=ZWN3AVQXYBELRTX7BTDMNTHT45HXD3U5AKGXNEFJM4Q5NRSKY6YA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.606134009 (444h51m57.376775344s) [noise=true , address=45.91.116.11:4123 , piece_id=WMPALJZGHW5TB7232YRTQ4DSJPFTTFTKMM63N2K6QIC6EDNVQYYQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.607118172 (444h51m57.377759507s) [noise=true , address=45.91.116.12:4259 , piece_id=OZMF3R53AUGH46LBKG53B7VLFGPZMMTNC5PDTVXF7U36O374WA4Q, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.618767719 (444h51m57.389409054s) [noise=true , address=77.90.31.18:4822 , piece_id=2A5A32R2PFRIOXF4RT7SPLDA7ZYBNQRZUEF3URXGCLFIN4CPKUGQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.623246287 (444h51m57.393887622s) [noise=true , address=77.90.31.15:4709 , piece_id=MNYBVFYI3APDTCXKNQ27PJB3NEOXK5NNJSTGO3L2KIGG7CQ6QHZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.625221406 (444h51m57.395862742s) [noise=true , address=45.91.116.8:4050 , piece_id=ZACWOJOY2IYEL4O6IGYLOG7KBADZ6QM26G246TSZVQ2YI6IEVBZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.650485021 (444h51m57.421126356s) [noise=true , address=77.90.31.11:4588 , piece_id=L5KFV5NCKUCLJ43ABMNDDNVTVU3LOKMKH7PVS7IV6BY4PFC5BIAQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.658800550 (444h51m57.429441885s) [noise=true , address=77.90.31.13:4637 , piece_id=PIVLKJ3G3UFVE4ZPKTN2OAR4CPUZDDQF62SOJFQFWFOCIRZKGK4A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.660254201 (444h51m57.430895526s) [noise=true , address=77.90.31.10:4538 , piece_id=ZG3GT4YWHXI2UIRBMOQRR6OE2IB3SM2PFW3JWCOBPLXHLBX42FEA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.663604691 (444h51m57.434246026s) [noise=true , address=77.90.31.19:4859 , piece_id=WHLA2M2RIHPBBFJRV4CXZERFKTC2HKVRPDLGJST3LWYXGNARBS6A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.667276506 (444h51m57.437917841s) [noise=true , address=77.90.31.21:4912 , piece_id=EP3FXEGB567M3G6ITDK3PQ2EP6474H33YPGJP5QAM2FVXOF5GP2A, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:02:22.777412707 (444h51m57.548054042s) [noise=true , address=79.127.219.38:4021 , piece_id=DXPMY6YAGV5HM7Q6SXDPGCFMDSVPLDJONZXDPHCSRGWEMFL5R5ZA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: <nil> Jul 11 22:03:19.601895201 (444h52m54.372536526s) [noise=true , address=104.36.82.4:4827 , piece_id=SZSP2QHIG4QKJULROHUJ6ESM7TCP6BWS7JYCLCOYKEHC5QZ7MW2Q, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1VXFTn1urnJ2qkUcDc5785hJ8L6bNVQTtgcgNeWLaZ9wEkKYVx, address:104.36.82.4:4827): protocol: write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection; write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45054->104.36.82.4:4827: use of closed network connection Jul 11 22:03:19.601925429 (444h52m54.372566764s) [noise=true , address=104.36.82.21:4490 , piece_id=TRIYXTVLJDLJLCGO64BUAJ2AXKNQPZIQNJHI2HBZGYYWQICYXEUA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:12DPUW1Ej5FLg9yTbhqdWBX8YAryWG82BnVegsCiTRX71pUA24n, address:104.36.82.21:4490): protocol: write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection; write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39984->104.36.82.21:4490: use of closed network connection Jul 11 22:03:19.601964724 (444h52m54.372606049s) [noise=true , address=104.36.82.2:4734 , piece_id=4PHLX43SMSCYJRC765X5DZDQTVKMLVZTYCFMIAQ4DRWIAYAJQ5TA, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:12GLb5LQje9wwivnj8Wj1tXcnS6mFb2Bco5saNwgmFJBVDujrv7, address:104.36.82.2:4734): protocol: write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection; write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47920->104.36.82.2:4734: use of closed network connection Jul 11 22:03:19.601995563 (444h52m54.372636898s) [noise=true , address=104.36.82.38:4692 , piece_id=PFGTVLSRSDMWTMOWGCYVFOUHSD3ERCW5PH4HP3ZMPUPEO34MA5UQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1n3SDzXGgXbvYBGtLdND9oy93AHNFokRnF6Yv3EvoMC9UgN2H9, address:104.36.82.38:4692): protocol: write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection; write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59348->104.36.82.38:4692: use of closed network connection Jul 11 22:03:19.602044546 (444h52m54.372685881s) [noise=true , address=104.36.82.17:4357 , piece_id=Z3345C4RUT7HBFVVK6MDZLUJP4PYFNW6POEGYILDJSHADFNWKKMQ, seg_pos={0 0}, upload=355807]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:104.36.82.17:4357): protocol: read tcp 188.166.225.170:60360->104.36.82.17:4357: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:03:19.602071177 (444h52m54.372712502s) [seg_pos={0 0}, upload=355807]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:03:19.829114955 (444h52m54.59975629s) [upload=355807]: Done uploading segments... Jul 11 22:03:19.829139813 (444h52m54.599781148s) [upload=355807]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.229373559s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.230745252s FAIL (2025-07-11 22:02:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== WGK53RKQLKDFSLVDJSWMGA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.568791992 (444h51m59.339433327s) [noise=true , address=104.36.82.15:4258 , piece_id=ALXEPUQVQSWZEGNLPFK3H55S6Z42E7FE6EQTI3LECPY5IPHJ73UA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.568824035 (444h51m59.33946537s) [noise=true , address=104.36.82.10:4078 , piece_id=3UV6KK7RYL2XO46QJWMUN5NNLBYJHQ2MTS3YFYKZB7TZDZJA6C2Q, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.569126292 (444h51m59.339767627s) [noise=true , address=77.90.31.11:4555 , piece_id=GDASX5WVKVLOMAJ2NK53XSOLJRXOME5YRQBNJRZL7YRICI7XWRMA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.569140730 (444h51m59.339782065s) [noise=true , address=104.36.82.10:4079 , piece_id=55HALCRJRT673S2HQSHESJ5BKTNRYONYV2W5ZKUZMUUJQNE2TIRA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.569411558 (444h51m59.340052893s) [noise=true , address=104.36.82.13:4187 , piece_id=FJJUYMKJNL2T6LKGFVEMPALWW54QCCQEKOQ6NO5YTL6S2CLG5YMA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.569428801 (444h51m59.340070136s) [noise=true , address=104.36.82.3:4767 , piece_id=IX66AO7YKYHMXX3EO6P53AT2EBQORJCYYTRBVR5S7UHW44QRP2KA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.569649553 (444h51m59.340290888s) [noise=true , address=104.36.82.12:4151 , piece_id=H3JAO6CUG24QR2W5NCULAM7MXKHKHCTVE3A4RJDCZYGV5UZZ4ADA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.569673439 (444h51m59.340314774s) [noise=true , address=104.36.82.13:4185 , piece_id=OK3F44KXRXRZ7GZI2ASIX3T3NPE7KAGYUXDEAII5TV5YL45G723Q, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.569899783 (444h51m59.340541118s) [noise=true , address=104.36.82.6:4876 , piece_id=T3TGDAR3U34WP7TU23SUWU6WAFW2LHHYEOBWG7IFCIO745M5U2NA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.569926773 (444h51m59.340568108s) [noise=true , address=77.90.31.12:4592 , piece_id=HOZYDZKYBS3VPHFHCR27AMPJ2ICYCQG5PASDS3N33XZGVXWSKYZA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.570126446 (444h51m59.340767771s) [noise=true , address=104.36.82.4:4803 , piece_id=E6QS2BWTRM5U5TEOECQQE6RWXMT2BYEPCQ6YQQH6BTSWRW3UJGVQ, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.570178907 (444h51m59.340820232s) [noise=true , address=104.36.82.11:4114 , piece_id=VQZ2SYJPH2VI7BXG5YLUUP2JBYU5I6M5GPOPAKJQJ5JEHJYAAB2A, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.570353931 (444h51m59.340995266s) [noise=true , address=104.36.82.2:4729 , piece_id=JKTLM67SW35P2SDKP3OSVVRKE6SO2LO2IPXUSSWAVIKN6Y73FDIA, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:24.570390531 (444h51m59.341031866s) [noise=true , address=45.91.116.11:4119 , piece_id=EY6US67W54NEGMKBBJAV7G6GHZOVWMAPXLMJD4HRBTOPPDA27L5A, seg_pos={0 0}, upload=355808]: Uploading piece... Jul 11 22:02:26.346493508 (444h52m1.117134843s) [upload=355808]: Next returned nil segment Jul 11 22:02:26.346512074 (444h52m1.117153439s) [upload=355808]: Waiting for error group managing segments... Jul 11 22:02:26.626713562 (444h52m1.397354897s) [noise=true , address=45.91.116.11:4119 , piece_id=EY6US67W54NEGMKBBJAV7G6GHZOVWMAPXLMJD4HRBTOPPDA27L5A, seg_pos={0 0}, upload=355808]: Done uploading piece. err: <nil> Jul 11 22:02:26.647204586 (444h52m1.417845921s) [noise=true , address=77.90.31.18:4806 , piece_id=XBI7FJXMXUX2QYMQGHHIJBUQGHALATSJQCZXA6MHU2S6OWPEOJOA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: <nil> Jul 11 22:02:26.658562946 (444h52m1.429204281s) [noise=true , address=77.90.31.11:4555 , piece_id=GDASX5WVKVLOMAJ2NK53XSOLJRXOME5YRQBNJRZL7YRICI7XWRMA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: <nil> Jul 11 22:02:26.667629552 (444h52m1.438270877s) [noise=true , address=77.90.31.12:4592 , piece_id=HOZYDZKYBS3VPHFHCR27AMPJ2ICYCQG5PASDS3N33XZGVXWSKYZA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: <nil> Jul 11 22:02:53.698527708 (444h52m28.469169043s) [noise=true , address=104.36.82.15:4258 , piece_id=ALXEPUQVQSWZEGNLPFK3H55S6Z42E7FE6EQTI3LECPY5IPHJ73UA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 188.166.225.170:34632->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698669659 (444h52m28.469310994s) [noise=true , address=104.36.82.10:4078 , piece_id=3UV6KK7RYL2XO46QJWMUN5NNLBYJHQ2MTS3YFYKZB7TZDZJA6C2Q, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: rpc: dial tcp 104.36.82.10:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698675831 (444h52m28.469317156s) [noise=true , address=104.36.82.2:4729 , piece_id=JKTLM67SW35P2SDKP3OSVVRKE6SO2LO2IPXUSSWAVIKN6Y73FDIA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: rpc: dial tcp 104.36.82.2:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698723472 (444h52m28.469364807s) [noise=true , address=104.36.82.13:4187 , piece_id=FJJUYMKJNL2T6LKGFVEMPALWW54QCCQEKOQ6NO5YTL6S2CLG5YMA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698754602 (444h52m28.469395937s) [noise=true , address=104.36.82.12:4151 , piece_id=H3JAO6CUG24QR2W5NCULAM7MXKHKHCTVE3A4RJDCZYGV5UZZ4ADA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698791151 (444h52m28.469432476s) [noise=true , address=104.36.82.6:4876 , piece_id=T3TGDAR3U34WP7TU23SUWU6WAFW2LHHYEOBWG7IFCIO745M5U2NA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698839424 (444h52m28.469480759s) [noise=true , address=104.36.82.3:4767 , piece_id=IX66AO7YKYHMXX3EO6P53AT2EBQORJCYYTRBVR5S7UHW44QRP2KA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698856397 (444h52m28.469497732s) [noise=true , address=104.36.82.11:4114 , piece_id=VQZ2SYJPH2VI7BXG5YLUUP2JBYU5I6M5GPOPAKJQJ5JEHJYAAB2A, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 188.166.225.170:38010->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698875533 (444h52m28.469516858s) [noise=true , address=104.36.82.13:4185 , piece_id=OK3F44KXRXRZ7GZI2ASIX3T3NPE7KAGYUXDEAII5TV5YL45G723Q, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: read tcp 188.166.225.170:48436->104.36.82.13:4185: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698894309 (444h52m28.469535634s) [noise=true , address=104.36.82.14:4222 , piece_id=XX6AG4SYDSG5VSQY23WQHZRYVGDI5IWGK53RKQLKDFSLVDJSWMGA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: read tcp 188.166.225.170:47404->104.36.82.14:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698930448 (444h52m28.469571783s) [noise=true , address=104.36.82.10:4079 , piece_id=55HALCRJRT673S2HQSHESJ5BKTNRYONYV2W5ZKUZMUUJQNE2TIRA, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: read tcp 188.166.225.170:43662->104.36.82.10:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698935458 (444h52m28.469576793s) [noise=true , address=104.36.82.4:4803 , piece_id=E6QS2BWTRM5U5TEOECQQE6RWXMT2BYEPCQ6YQQH6BTSWRW3UJGVQ, seg_pos={0 0}, upload=355808]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: read tcp 188.166.225.170:43440->104.36.82.4:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:53.698950106 (444h52m28.469591431s) [seg_pos={0 0}, upload=355808]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 11 22:02:53.926983897 (444h52m28.697625233s) [upload=355808]: Done uploading segments... Jul 11 22:02:53.927006931 (444h52m28.697648266s) [upload=355808]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.230758819s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [XX.........................................................X] duration: 1m0.000934517s FAIL (2025-07-11 22:02:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000956841s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XX......................................................X.XX] duration: 30.001313582s FAIL (2025-07-11 22:02:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001324163s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.243057443s FAIL (2025-07-11 22:02:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 1:53.878522822 (444h51m28.649164157s) [upload=355802]: Got next segment. Inline: false Jul 11 22:01:53.878563218 (444h51m28.649204553s) [upload=355802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:01:53.878567787 (444h51m28.649209122s) [upload=355802]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:01:54.114543051 (444h51m28.885184386s) [seg_pos={0 0}, upload=355802]: Begin upload segment... Jul 11 22:01:54.114587656 (444h51m28.885228991s) [seg_pos={0 0}, upload=355802]: Done begin upload segment. Jul 11 22:01:54.114609588 (444h51m28.885250923s) [noise=true , address=104.36.82.16:4294 , piece_id=L3DXTMTRK6SAH757N72QY6C2QA7SNWBJHF5BITFS3WZYNSB7VCZA, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.114665836 (444h51m28.885307171s) [noise=true , address=104.36.82.15:4257 , piece_id=HZQ5UWNYJ3ZRPGT7BXOS7OG7Y5PD5WLBIV2UJSRUPBZ4CFQL7SVQ, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.114974787 (444h51m28.885616122s) [noise=true , address=104.36.82.11:4113 , piece_id=TF4UNATZI6IPOKJGRWGXDHLM7UTYVKNW7QNI2MNDLOUQIQ5UUG7Q, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.115095168 (444h51m28.885736503s) [noise=true , address=77.90.31.14:4662 , piece_id=VKLQSZ37FEWB7G7GJQBFSR2U5WMQPEBTZY4RMG7DEUWRZMIJNJRA, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.115249833 (444h51m28.885891168s) [noise=true , address=77.90.31.17:4771 , piece_id=AAQD2A4X72SDT3HGEWVTYYWPUDDUZ6NSJFKECXOA26FFS5DE5GPA, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.115375794 (444h51m28.886017129s) [noise=true , address=104.36.82.2:4732 , piece_id=BQ25N2ZG55HACAPUOGA3HXZPUZFFNYOUQD2SC3Z64BIZHR3EYXJA, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.115519509 (444h51m28.886160844s) [noise=true , address=45.91.116.5:4298 , piece_id=J5CNKSK3GO545TIJJVSE3Y4T7UQDDT57WCMI4IXC2P4SZAM4NOJQ, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.115643988 (444h51m28.886285323s) [noise=true , address=77.90.31.21:4878 , piece_id=A4VOXA24W3HOPJNMS5RVQM3FSQW4JH52K4RDHCXXSN7EKOBUVQTA, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.115767193 (444h51m28.886408528s) [noise=true , address=104.36.82.10:4077 , piece_id=ALIVOO3BRXUGZYPDPP3C7TFSYF4TV434MSSAPU2US2K64IQWCRRA, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.115891060 (444h51m28.886532395s) [noise=true , address=45.91.116.6:4973 , piece_id=ISTUDPXTJBNU7GNLXGIJ3FYXOQYC7XIMWIBJGCTD2IKZUBP4Q65Q, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.116015809 (444h51m28.886657144s) [noise=true , address=77.90.31.12:4590 , piece_id=FBYXMTKJLET2PJPLBKCWHTRUCRFX43MHEGHPI7ARZXAORQYDXMKQ, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.116248194 (444h51m28.886889519s) [noise=true , address=104.36.82.12:4150 , piece_id=MNDAOL2WSLFSRWSQPLS7XTMNDLTSAPGWYLL23ZT3DQDNF7GSUQIQ, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.116428358 (444h51m28.887069693s) [noise=true , address=104.36.82.9:4006 , piece_id=7STAD2YP6ENH2YSLEX7I3BSOYQ2BWMXMTBY46TUYJWSH6G6CLXXQ, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.116468115 (444h51m28.88710945s) [noise=true , address=104.36.82.4:4804 , piece_id=GQEQGREFY7TE4WRIU3X3UB3QJQB4K5YPORU7RUBKIVEBSVU6XSVA, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.116788507 (444h51m28.887429842s) [noise=true , address=45.91.116.4:4938 , piece_id=YM63VGGXN4NLFG7RSZFZNAPD2YRLN4SHREBCBFBVZWUXBD45ROGA, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:54.116856888 (444h51m28.887498213s) [noise=true , address=104.36.82.14:4221 , piece_id=UY2KR5L4PFJVXK4B4PQ6WKAAAC4PN7CZHGST2FXDBSV3QLNSE5YQ, seg_pos={0 0}, upload=355802]: Uploading piece... Jul 11 22:01:55.868710675 (444h51m30.63935201s) [upload=355802]: Next returned nil segment Jul 11 22:01:55.868731365 (444h51m30.6393727s) [upload=355802]: Waiting for error group managing segments... Jul 11 22:01:56.171376251 (444h51m30.942017586s) [noise=true , address=45.91.116.5:4298 , piece_id=J5CNKSK3GO545TIJJVSE3Y4T7UQDDT57WCMI4IXC2P4SZAM4NOJQ, seg_pos={0 0}, upload=355802]: Done uploading piece. err: <nil> Jul 11 22:01:56.172505903 (444h51m30.943147238s) [noise=true , address=45.91.116.6:4973 , piece_id=ISTUDPXTJBNU7GNLXGIJ3FYXOQYC7XIMWIBJGCTD2IKZUBP4Q65Q, seg_pos={0 0}, upload=355802]: Done uploading piece. err: <nil> Jul 11 22:01:56.175778223 (444h51m30.946419558s) [noise=true , address=45.91.116.4:4938 , piece_id=YM63VGGXN4NLFG7RSZFZNAPD2YRLN4SHREBCBFBVZWUXBD45ROGA, seg_pos={0 0}, upload=355802]: Done uploading piece. err: <nil> Jul 11 22:01:56.196066919 (444h51m30.966708254s) [noise=true , address=77.90.31.17:4771 , piece_id=AAQD2A4X72SDT3HGEWVTYYWPUDDUZ6NSJFKECXOA26FFS5DE5GPA, seg_pos={0 0}, upload=355802]: Done uploading piece. err: <nil> Jul 11 22:01:56.208818205 (444h51m30.97945953s) [noise=true , address=77.90.31.14:4662 , piece_id=VKLQSZ37FEWB7G7GJQBFSR2U5WMQPEBTZY4RMG7DEUWRZMIJNJRA, seg_pos={0 0}, upload=355802]: Done uploading piece. err: <nil> Jul 11 22:01:56.210672202 (444h51m30.981313537s) [noise=true , address=77.90.31.12:4590 , piece_id=FBYXMTKJLET2PJPLBKCWHTRUCRFX43MHEGHPI7ARZXAORQYDXMKQ, seg_pos={0 0}, upload=355802]: Done uploading piece. err: <nil> Jul 11 22:01:56.214365618 (444h51m30.985006954s) [noise=true , address=77.90.31.21:4878 , piece_id=A4VOXA24W3HOPJNMS5RVQM3FSQW4JH52K4RDHCXXSN7EKOBUVQTA, seg_pos={0 0}, upload=355802]: Done uploading piece. err: <nil> Jul 11 22:02:23.236440364 (444h51m58.007081699s) [noise=true , address=104.36.82.16:4294 , piece_id=L3DXTMTRK6SAH757N72QY6C2QA7SNWBJHF5BITFS3WZYNSB7VCZA, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: rpc: dial tcp 104.36.82.16:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:23.236541637 (444h51m58.007182972s) [noise=true , address=104.36.82.9:4006 , piece_id=7STAD2YP6ENH2YSLEX7I3BSOYQ2BWMXMTBY46TUYJWSH6G6CLXXQ, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: rpc: dial tcp 104.36.82.9:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:23.236631830 (444h51m58.007273165s) [noise=true , address=104.36.82.10:4077 , piece_id=ALIVOO3BRXUGZYPDPP3C7TFSYF4TV434MSSAPU2US2K64IQWCRRA, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: write tcp 188.166.225.170:54802->104.36.82.10:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:23.236712434 (444h51m58.007353769s) [noise=true , address=104.36.82.4:4804 , piece_id=GQEQGREFY7TE4WRIU3X3UB3QJQB4K5YPORU7RUBKIVEBSVU6XSVA, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: rpc: dial tcp 104.36.82.4:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:23.236783540 (444h51m58.007424905s) [noise=true , address=104.36.82.12:4150 , piece_id=MNDAOL2WSLFSRWSQPLS7XTMNDLTSAPGWYLL23ZT3DQDNF7GSUQIQ, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: rpc: dial tcp 104.36.82.12:4150: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:23.236804560 (444h51m58.007445885s) [noise=true , address=104.36.82.11:4113 , piece_id=TF4UNATZI6IPOKJGRWGXDHLM7UTYVKNW7QNI2MNDLOUQIQ5UUG7Q, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: context deadline exceeded; context deadline exceeded Jul 11 22:02:23.236869013 (444h51m58.007510348s) [noise=true , address=104.36.82.2:4732 , piece_id=BQ25N2ZG55HACAPUOGA3HXZPUZFFNYOUQD2SC3Z64BIZHR3EYXJA, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:23.236878622 (444h51m58.007519977s) [noise=true , address=104.36.82.14:4221 , piece_id=UY2KR5L4PFJVXK4B4PQ6WKAAAC4PN7CZHGST2FXDBSV3QLNSE5YQ, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: rpc: dial tcp 104.36.82.14:4221: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:23.236934960 (444h51m58.007576315s) [noise=true , address=104.36.82.15:4257 , piece_id=HZQ5UWNYJ3ZRPGT7BXOS7OG7Y5PD5WLBIV2UJSRUPBZ4CFQL7SVQ, seg_pos={0 0}, upload=355802]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:55342->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:23.236972221 (444h51m58.007613586s) [seg_pos={0 0}, upload=355802]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 11 22:02:23.477311968 (444h51m58.247953303s) [upload=355802]: Done uploading segments... Jul 11 22:02:23.477347005 (444h51m58.24798834s) [upload=355802]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.243082409s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXX......................................................X.X] duration: 30.000405492s FAIL (2025-07-11 22:02:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000423826s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXX......................................................X.] duration: 10.357512398s FAIL (2025-07-11 22:00:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.357626769s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 7.76357211s FAIL (2025-07-11 22:00:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.76359344s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXX......................................................X] duration: 30.000547602s FAIL (2025-07-11 22:00:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000568592s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.708273509s FAIL (2025-07-11 21:59:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.708298415s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XXX......................................................] duration: 6.353902286s FAIL (2025-07-11 21:59:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.35392521s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 7.824525256s FAIL (2025-07-11 21:59:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.824546315s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 5.531906326s FAIL (2025-07-11 21:58:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.532879087s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XXX.....................................................] duration: 6.046641811s FAIL (2025-07-11 21:58:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.046663372s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 5.566543573s FAIL (2025-07-11 21:57:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.566564043s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XXX....................................................] duration: 6.080053813s FAIL (2025-07-11 21:57:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.080076055s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 5.70600528s FAIL (2025-07-11 21:57:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.706030898s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XXX...................................................] duration: 6.166017968s FAIL (2025-07-11 21:56:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.166040162s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 5.620365464s FAIL (2025-07-11 21:56:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.620380995s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 14.160238512s FAIL (2025-07-11 21:56:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.160259251s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......XXX..................................................] duration: 8.002099136s FAIL (2025-07-11 21:55:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.002118263s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 11.716970291s FAIL (2025-07-11 21:55:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.716992374s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........XXX.................................................] duration: 6.169341158s FAIL (2025-07-11 21:55:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.169361467s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........XXX................................................] duration: 6.013223554s FAIL (2025-07-11 21:55:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.013245446s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 5.463096331s FAIL (2025-07-11 21:54:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.463145014s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........XXX...............................................] duration: 6.022494004s FAIL (2025-07-11 21:54:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.022508132s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........XXX..............................................] duration: 6.055743183s FAIL (2025-07-11 21:54:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.056052976s edge gateway us-select-1

1
...
8208
8209
8210
8211
8212
...
8559