Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 5.953677617s FAIL (2025-07-26 20:38:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.953704939s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 5.918325226s FAIL (2025-07-26 20:38:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.918339964s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.972132662s FAIL (2025-07-26 20:38:10 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.972163211s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XXX............] duration: 6.061359452s FAIL (2025-07-26 20:38:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.061385642s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X..............X.............X.............] duration: 6.289766031s FAIL (2025-07-26 20:37:59 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.289790398s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 6.159206604s FAIL (2025-07-26 20:37:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.15921981s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.830621089s FAIL (2025-07-26 20:37:08 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.8306471s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration: 6.376167944s FAIL (2025-07-26 20:37:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.376199845s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 6.015110527s FAIL (2025-07-26 20:36:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.015141566s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X..............X.............X............] duration: 6.319854238s FAIL (2025-07-26 20:36:52 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.319881621s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.925420748s FAIL (2025-07-26 20:36:50 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.925447479s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.875946611s FAIL (2025-07-26 20:36:21 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.87596692s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X..............X.............X...........] duration: 6.421493867s FAIL (2025-07-26 20:36:08 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.421524596s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 5.99573741s FAIL (2025-07-26 20:36:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.995802083s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration: 5.901282438s FAIL (2025-07-26 20:36:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.901312536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.837578754s FAIL (2025-07-26 20:35:38 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.837605674s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................XXX.........] duration: 6.339233621s FAIL (2025-07-26 20:35:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.339254721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X..............X.............X..........] duration: 5.948000284s FAIL (2025-07-26 20:35:10 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.948026585s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 6.087243697s FAIL (2025-07-26 20:34:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.08726051s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XXX........] duration: 6.600127274s FAIL (2025-07-26 20:34:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.600147692s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.907275173s FAIL (2025-07-26 20:34:42 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.907302886s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X..............X.............X.........] duration: 5.83619558s FAIL (2025-07-26 20:34:41 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.836251786s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XXX.......] duration: 6.040679446s FAIL (2025-07-26 20:34:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.040709743s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.943216758s FAIL (2025-07-26 20:34:15 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.943246096s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XXX......] duration: 6.282496114s FAIL (2025-07-26 20:33:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.282523727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.922676418s FAIL (2025-07-26 20:33:52 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.922710424s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 6.215507292s FAIL (2025-07-26 20:33:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.21553227s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XXX.....] duration: 6.040052105s FAIL (2025-07-26 20:33:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.040074727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X..............X.............X........] duration: 6.401086568s FAIL (2025-07-26 20:33:28 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.401111777s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 6.315415676s FAIL (2025-07-26 20:33:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.315439302s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XXX....] duration: 6.297646549s FAIL (2025-07-26 20:33:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.297667069s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X..............X.............X.......] duration: 6.123044348s FAIL (2025-07-26 20:33:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.123069195s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.902858242s FAIL (2025-07-26 20:32:58 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.902911783s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X..............X.............X......] duration: 6.688250631s FAIL (2025-07-26 20:32:32 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.688276381s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.890045115s FAIL (2025-07-26 20:32:23 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.890072979s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 5.786847079s FAIL (2025-07-26 20:32:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.786868199s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XXX...] duration: 6.666776276s FAIL (2025-07-26 20:32:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.66679957s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X..............X.............X.....] duration: 6.433309634s FAIL (2025-07-26 20:32:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.433355673s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.849086781s FAIL (2025-07-26 20:31:41 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.849116077s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.021847452s FAIL (2025-07-26 20:31:24 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 2.021879264s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XXX..] duration: 6.276648397s FAIL (2025-07-26 20:31:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.276678674s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 6.063746161s FAIL (2025-07-26 20:31:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.063774825s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.889467741s FAIL (2025-07-26 20:30:53 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 1.889494713s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X..............X.............X....] duration: 6.488068642s FAIL (2025-07-26 20:30:51 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.488103218s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 6.12774504s FAIL (2025-07-26 20:30:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.127766983s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X..............X.............X...] duration: 6.413757552s FAIL (2025-07-26 20:30:21 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.413782078s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 6.241278687s FAIL (2025-07-26 20:30:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== iece_id=JIGNC43VKD3YVARU6BHSPMLAMT3FRSKLSPNAURBYNTFANBMCJ7RA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: write tcp 188.166.225.170:38056->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255340599 (56h52m56.977903639s) [noise=true , address=45.91.116.14:4189 , piece_id=H6XUSMWDVUJXANZVSQETB7Y44KEBB574DCOHEXWBIBOJXXH5Y7OA, seg_pos={0 0}, upload=54593]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: rpc: dial tcp 45.91.116.14:4189: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:47.255384323 (56h52m56.977947363s) [seg_pos={0 0}, upload=54593]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 26 20:28:47.486510951 (56h52m57.209073991s) [upload=54593]: Done uploading segments... Jul 26 20:28:47.486551077 (56h52m57.209114107s) [upload=54593]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Jul 26 20:28:50.713181364 (56h53m0.435744404s) [upload=54599]: Starting upload Jul 26 20:28:50.713248432 (56h53m0.435811472s) [upload=54599]: Done starting upload Jul 26 20:28:50.713312314 (56h53m0.435875354s) [upload=54599]: Uploading segments... Jul 26 20:28:50.713452573 (56h53m0.436015603s) [upload=54599]: Got next segment. Inline: false Jul 26 20:28:50.713476478 (56h53m0.436039519s) [upload=54599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:50.713480777 (56h53m0.436043817s) [upload=54599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:50.949390366 (56h53m0.671953466s) [seg_pos={0 0}, upload=54599]: Begin upload segment... Jul 26 20:28:50.949444028 (56h53m0.672007078s) [seg_pos={0 0}, upload=54599]: Done begin upload segment. Jul 26 20:28:50.949500426 (56h53m0.672063466s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949629433 (56h53m0.672192463s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.949977509 (56h53m0.672540539s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950235172 (56h53m0.672798212s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950257304 (56h53m0.672820344s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950548682 (56h53m0.673111722s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950751770 (56h53m0.6733148s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950764715 (56h53m0.673327755s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.950977603 (56h53m0.673540643s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951011087 (56h53m0.673574127s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951350225 (56h53m0.673913265s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951743598 (56h53m0.674306628s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.951870891 (56h53m0.674433931s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.952532227 (56h53m0.675095257s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.953348769 (56h53m0.675911809s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:50.956236154 (56h53m0.678799184s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Uploading piece... Jul 26 20:28:54.584558914 (56h53m4.307121954s) [upload=54599]: Next returned nil segment Jul 26 20:28:54.584581508 (56h53m4.307144548s) [upload=54599]: Waiting for error group managing segments... Jul 26 20:28:55.375834954 (56h53m5.098398034s) [noise=true , address=77.90.31.12:4590 , piece_id=ARHJG7RVPKZVH6CUXFOPEDKB2AJIPMFAJVG25Q2TK6LHD4IBWWWA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:55.754417801 (56h53m5.476980841s) [noise=true , address=77.90.31.11:4554 , piece_id=BUYLPY7NLDS5FSYPDDXYKUZ33PNEHXN4BHO7XY27JJTP7YGHAZVQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.120567801 (56h53m5.843130841s) [noise=true , address=77.90.31.17:4772 , piece_id=S43R2DCSBAEYSAL3BNCQ7ALNLHPSZCA7VZ5YQHOSMIJQAYM2DECQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.147135805 (56h53m5.869698845s) [noise=true , address=77.90.31.18:4805 , piece_id=P7KQ5SZ2ISZOCS3MCVWX2YPOXNG24K5ZTAARIDVQ7PAZXVGBMNFQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:56.217572301 (56h53m5.940135341s) [noise=true , address=77.90.31.13:4628 , piece_id=A6XWIROEGOZXJC25NNJAJGC6NNFZV6K4GDLWECXWF6P5H4GIMHAQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.106221031 (56h53m7.828784101s) [noise=true , address=77.90.31.12:4589 , piece_id=GO5HZYFCBDNKSE3DMK6BYNRQK6QSHTFN25NPHGOVRX3UCPLDGYNQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:28:58.696825733 (56h53m8.419388764s) [noise=true , address=77.90.31.17:4771 , piece_id=IL4AGLMJUO4FOI2DFWTVESJVBXQESOPBPCQPY72IATQXJ4KZ6BCA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.838197092 (56h53m18.560760122s) [noise=true , address=45.91.116.14:4191 , piece_id=OYBQLD4KTXNQ4YAPBXJPRREOZNM3IH6GDUAJWMIQUNGMPHEKDNSA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.840921915 (56h53m18.563484955s) [noise=true , address=45.91.116.4:4939 , piece_id=4I4GPIZZLQ3BSRZSMJ2NL6QXXYCZ7NJZY3EA6GH6K2OLA7PET55Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.944779827 (56h53m18.667342867s) [noise=true , address=45.91.116.14:4190 , piece_id=OWBIICTMFAUPD7JZHZSQ32PL723LKT6JIXQTUAJEYIQO625JBX7Q, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:08.968949992 (56h53m18.691513032s) [noise=true , address=45.91.116.8:4047 , piece_id=3F4C2BQNUZVNEN2AISGVHD3KML5VNDIAIXDWOQMPST5TNBDMTSMA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:09.074863428 (56h53m18.797426468s) [noise=true , address=45.91.116.7:4011 , piece_id=KHVS7FKAGCRNTYVWEYGMXRSICPH3KOC4JXQJERWRC6X7PBRWJ24A, seg_pos={0 0}, upload=54599]: Done uploading piece. err: <nil> Jul 26 20:29:20.062079499 (56h53m29.784642579s) [noise=true , address=104.36.82.7:4910 , piece_id=SEHRW22C4QUPD2VO2CFPXITZPKWPHF45656ZH6L4O57ZQCP7JYZA, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062272147 (56h53m29.784835218s) [noise=true , address=104.36.82.13:4187 , piece_id=JVAKXUM3WAGXXIM4EC23DE43HXDAGPRLZQH2FKQI44EQ76633DTQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062359615 (56h53m29.784922655s) [noise=true , address=104.36.82.9:4005 , piece_id=OJUNB6IYWDBPXHKJYNFYMSEE3A6Y6JNYX44FBRJKTJPHJPCCU6KQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062433816 (56h53m29.784996856s) [noise=true , address=104.36.82.10:4078 , piece_id=G366YAGZHU2V64PJP7KGFDSAVE5GH3IOXZPXFVNO5QO6UDBDMNQQ, seg_pos={0 0}, upload=54599]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:29:20.062447683 (56h53m29.785010723s) [seg_pos={0 0}, upload=54599]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 20:29:20.289763289 (56h53m30.012326339s) [upload=54599]: Done uploading segments... Jul 26 20:29:20.289790922 (56h53m30.012353962s) [upload=54599]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.241312021s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.073412209s FAIL (2025-07-26 20:30:05 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/1K") Duration: 2.073442327s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 6.191737627s FAIL (2025-07-26 20:30:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.191767874s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X..............X.............X..] duration: 8.428737797s FAIL (2025-07-26 20:29:26 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 8) Duration: 8.428754309s core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)