Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 27.197640544s FAIL (2025-07-09 23:08:18 UTC) orbiter warning: context deadline exceeded Duration: 27.198024046s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.X...............................X..X.......X.......X.......] duration: 49.752184009s FAIL (2025-07-09 23:08:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 0 (397h51m47.070257915s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.299863513 (397h51m47.070504848s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300155401 (397h51m47.070796736s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300500541 (397h51m47.071141886s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300707928 (397h51m47.071349263s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300895386 (397h51m47.071536712s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301093135 (397h51m47.07173447s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301289260 (397h51m47.071930595s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301483822 (397h51m47.072125147s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301668996 (397h51m47.072310332s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301881594 (397h51m47.072522929s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302036139 (397h51m47.072677484s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302541906 (397h51m47.073183241s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302904690 (397h51m47.073546025s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306087348 (397h51m47.076728683s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306494538 (397h51m47.077135883s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306742372 (397h51m47.077383707s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:13.699557207 (397h51m48.470198542s) [upload=318314]: Next returned nil segment Jul 9 23:02:13.699575412 (397h51m48.470216747s) [upload=318314]: Waiting for error group managing segments... Jul 9 23:02:14.004198502 (397h51m48.774839857s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.317361159 (397h51m49.088002494s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.318855398 (397h51m49.089496723s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.321324141 (397h51m49.091965476s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.322031012 (397h51m49.092672357s) [noise=true , address=77.90.31.12:4602 , piece_id=YALPUB6SRLSX54PKVC4UOKREEAXYZ22JE3FDX4OS3I6RUC7B5KOQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323073236 (397h51m49.093714572s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323756533 (397h51m49.094397868s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.326098634 (397h51m49.096739969s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.329541540 (397h51m49.100182875s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:22.918099454 (397h51m57.688740789s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:03:11.197265107 (397h52m45.967906442s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj, address:45.91.95.2:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197291919 (397h52m45.967933254s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12Yb4nEC9PfMTDs4iPU4WmSCHJrALQSDzKSwyD2z5g6WxM5Q5cB, address:45.91.95.3:4009): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197383253 (397h52m45.968024588s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection Jul 9 23:03:11.197407600 (397h52m45.968048935s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12X8Et5FNyAsE7ifEckfQemZkfnvYUuZ2Rn2xdxNxhmCmuDST2M, address:45.91.95.5:4022): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197423410 (397h52m45.968064745s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY, address:45.91.95.4:4001): protocol: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection Jul 9 23:03:11.197480189 (397h52m45.968121524s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197527659 (397h52m45.968168994s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197541516 (397h52m45.968182841s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: expected piece hash; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; EOF Jul 9 23:03:11.197562597 (397h52m45.968203932s) [seg_pos={0 0}, upload=318314]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 9 23:03:11.438173501 (397h52m46.208814836s) [upload=318314]: Done uploading segments... Jul 9 23:03:11.438197957 (397h52m46.208839292s) [upload=318314]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 49.752212203s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X...............X...X.......................] duration: 6.171527616s FAIL (2025-07-09 23:08:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.17155069s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X...............X...X......................] duration: 6.041098182s FAIL (2025-07-09 23:07:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.041124904s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 12.074938396s FAIL (2025-07-09 23:06:59 UTC) orbiter warning: context deadline exceeded Duration: 12.074958605s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 55.254506965s FAIL (2025-07-09 23:06:43 UTC) orbiter warning: context deadline exceeded Duration: 55.254537874s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [..X...............................X..X.......X.......X......] duration: 10.670719446s FAIL (2025-07-09 23:06:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 0 (397h51m47.070257915s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.299863513 (397h51m47.070504848s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300155401 (397h51m47.070796736s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300500541 (397h51m47.071141886s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300707928 (397h51m47.071349263s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300895386 (397h51m47.071536712s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301093135 (397h51m47.07173447s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301289260 (397h51m47.071930595s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301483822 (397h51m47.072125147s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301668996 (397h51m47.072310332s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301881594 (397h51m47.072522929s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302036139 (397h51m47.072677484s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302541906 (397h51m47.073183241s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302904690 (397h51m47.073546025s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306087348 (397h51m47.076728683s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306494538 (397h51m47.077135883s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306742372 (397h51m47.077383707s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:13.699557207 (397h51m48.470198542s) [upload=318314]: Next returned nil segment Jul 9 23:02:13.699575412 (397h51m48.470216747s) [upload=318314]: Waiting for error group managing segments... Jul 9 23:02:14.004198502 (397h51m48.774839857s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.317361159 (397h51m49.088002494s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.318855398 (397h51m49.089496723s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.321324141 (397h51m49.091965476s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.322031012 (397h51m49.092672357s) [noise=true , address=77.90.31.12:4602 , piece_id=YALPUB6SRLSX54PKVC4UOKREEAXYZ22JE3FDX4OS3I6RUC7B5KOQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323073236 (397h51m49.093714572s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323756533 (397h51m49.094397868s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.326098634 (397h51m49.096739969s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.329541540 (397h51m49.100182875s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:22.918099454 (397h51m57.688740789s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:03:11.197265107 (397h52m45.967906442s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj, address:45.91.95.2:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197291919 (397h52m45.967933254s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12Yb4nEC9PfMTDs4iPU4WmSCHJrALQSDzKSwyD2z5g6WxM5Q5cB, address:45.91.95.3:4009): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197383253 (397h52m45.968024588s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection Jul 9 23:03:11.197407600 (397h52m45.968048935s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12X8Et5FNyAsE7ifEckfQemZkfnvYUuZ2Rn2xdxNxhmCmuDST2M, address:45.91.95.5:4022): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197423410 (397h52m45.968064745s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY, address:45.91.95.4:4001): protocol: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection Jul 9 23:03:11.197480189 (397h52m45.968121524s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197527659 (397h52m45.968168994s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197541516 (397h52m45.968182841s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: expected piece hash; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; EOF Jul 9 23:03:11.197562597 (397h52m45.968203932s) [seg_pos={0 0}, upload=318314]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 9 23:03:11.438173501 (397h52m46.208814836s) [upload=318314]: Done uploading segments... Jul 9 23:03:11.438197957 (397h52m46.208839292s) [upload=318314]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 10.67074185s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [...X...............................X..X.......X.......X.....] duration: 12.928200573s FAIL (2025-07-09 23:06:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 0 (397h51m47.070257915s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.299863513 (397h51m47.070504848s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300155401 (397h51m47.070796736s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300500541 (397h51m47.071141886s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300707928 (397h51m47.071349263s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300895386 (397h51m47.071536712s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301093135 (397h51m47.07173447s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301289260 (397h51m47.071930595s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301483822 (397h51m47.072125147s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301668996 (397h51m47.072310332s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301881594 (397h51m47.072522929s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302036139 (397h51m47.072677484s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302541906 (397h51m47.073183241s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302904690 (397h51m47.073546025s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306087348 (397h51m47.076728683s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306494538 (397h51m47.077135883s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306742372 (397h51m47.077383707s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:13.699557207 (397h51m48.470198542s) [upload=318314]: Next returned nil segment Jul 9 23:02:13.699575412 (397h51m48.470216747s) [upload=318314]: Waiting for error group managing segments... Jul 9 23:02:14.004198502 (397h51m48.774839857s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.317361159 (397h51m49.088002494s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.318855398 (397h51m49.089496723s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.321324141 (397h51m49.091965476s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.322031012 (397h51m49.092672357s) [noise=true , address=77.90.31.12:4602 , piece_id=YALPUB6SRLSX54PKVC4UOKREEAXYZ22JE3FDX4OS3I6RUC7B5KOQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323073236 (397h51m49.093714572s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323756533 (397h51m49.094397868s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.326098634 (397h51m49.096739969s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.329541540 (397h51m49.100182875s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:22.918099454 (397h51m57.688740789s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:03:11.197265107 (397h52m45.967906442s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj, address:45.91.95.2:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197291919 (397h52m45.967933254s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12Yb4nEC9PfMTDs4iPU4WmSCHJrALQSDzKSwyD2z5g6WxM5Q5cB, address:45.91.95.3:4009): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197383253 (397h52m45.968024588s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection Jul 9 23:03:11.197407600 (397h52m45.968048935s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12X8Et5FNyAsE7ifEckfQemZkfnvYUuZ2Rn2xdxNxhmCmuDST2M, address:45.91.95.5:4022): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197423410 (397h52m45.968064745s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY, address:45.91.95.4:4001): protocol: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection Jul 9 23:03:11.197480189 (397h52m45.968121524s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197527659 (397h52m45.968168994s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197541516 (397h52m45.968182841s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: expected piece hash; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; EOF Jul 9 23:03:11.197562597 (397h52m45.968203932s) [seg_pos={0 0}, upload=318314]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 9 23:03:11.438173501 (397h52m46.208814836s) [upload=318314]: Done uploading segments... Jul 9 23:03:11.438197957 (397h52m46.208839292s) [upload=318314]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 12.928230288s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X...............X...X.....................] duration: 11.215994526s FAIL (2025-07-09 23:06:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.216013773s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [....X...............................X..X.......X.......X....] duration: 16.347992398s FAIL (2025-07-09 23:05:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 0 (397h51m47.070257915s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.299863513 (397h51m47.070504848s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300155401 (397h51m47.070796736s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300500541 (397h51m47.071141886s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300707928 (397h51m47.071349263s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300895386 (397h51m47.071536712s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301093135 (397h51m47.07173447s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301289260 (397h51m47.071930595s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301483822 (397h51m47.072125147s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301668996 (397h51m47.072310332s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301881594 (397h51m47.072522929s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302036139 (397h51m47.072677484s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302541906 (397h51m47.073183241s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302904690 (397h51m47.073546025s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306087348 (397h51m47.076728683s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306494538 (397h51m47.077135883s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306742372 (397h51m47.077383707s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:13.699557207 (397h51m48.470198542s) [upload=318314]: Next returned nil segment Jul 9 23:02:13.699575412 (397h51m48.470216747s) [upload=318314]: Waiting for error group managing segments... Jul 9 23:02:14.004198502 (397h51m48.774839857s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.317361159 (397h51m49.088002494s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.318855398 (397h51m49.089496723s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.321324141 (397h51m49.091965476s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.322031012 (397h51m49.092672357s) [noise=true , address=77.90.31.12:4602 , piece_id=YALPUB6SRLSX54PKVC4UOKREEAXYZ22JE3FDX4OS3I6RUC7B5KOQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323073236 (397h51m49.093714572s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323756533 (397h51m49.094397868s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.326098634 (397h51m49.096739969s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.329541540 (397h51m49.100182875s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:22.918099454 (397h51m57.688740789s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:03:11.197265107 (397h52m45.967906442s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj, address:45.91.95.2:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197291919 (397h52m45.967933254s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12Yb4nEC9PfMTDs4iPU4WmSCHJrALQSDzKSwyD2z5g6WxM5Q5cB, address:45.91.95.3:4009): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197383253 (397h52m45.968024588s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection Jul 9 23:03:11.197407600 (397h52m45.968048935s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12X8Et5FNyAsE7ifEckfQemZkfnvYUuZ2Rn2xdxNxhmCmuDST2M, address:45.91.95.5:4022): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197423410 (397h52m45.968064745s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY, address:45.91.95.4:4001): protocol: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection Jul 9 23:03:11.197480189 (397h52m45.968121524s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197527659 (397h52m45.968168994s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197541516 (397h52m45.968182841s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: expected piece hash; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; EOF Jul 9 23:03:11.197562597 (397h52m45.968203932s) [seg_pos={0 0}, upload=318314]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 9 23:03:11.438173501 (397h52m46.208814836s) [upload=318314]: Done uploading segments... Jul 9 23:03:11.438197957 (397h52m46.208839292s) [upload=318314]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 16.348009201s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 17.762997068s FAIL (2025-07-09 23:05:24 UTC) orbiter warning: context deadline exceeded Duration: 17.763019843s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.....X...............................X..X.......X.......X...] duration: 21.976966733s FAIL (2025-07-09 23:05:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 0 (397h51m47.070257915s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.299863513 (397h51m47.070504848s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300155401 (397h51m47.070796736s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300500541 (397h51m47.071141886s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300707928 (397h51m47.071349263s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300895386 (397h51m47.071536712s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301093135 (397h51m47.07173447s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301289260 (397h51m47.071930595s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301483822 (397h51m47.072125147s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301668996 (397h51m47.072310332s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301881594 (397h51m47.072522929s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302036139 (397h51m47.072677484s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302541906 (397h51m47.073183241s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302904690 (397h51m47.073546025s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306087348 (397h51m47.076728683s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306494538 (397h51m47.077135883s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306742372 (397h51m47.077383707s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:13.699557207 (397h51m48.470198542s) [upload=318314]: Next returned nil segment Jul 9 23:02:13.699575412 (397h51m48.470216747s) [upload=318314]: Waiting for error group managing segments... Jul 9 23:02:14.004198502 (397h51m48.774839857s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.317361159 (397h51m49.088002494s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.318855398 (397h51m49.089496723s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.321324141 (397h51m49.091965476s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.322031012 (397h51m49.092672357s) [noise=true , address=77.90.31.12:4602 , piece_id=YALPUB6SRLSX54PKVC4UOKREEAXYZ22JE3FDX4OS3I6RUC7B5KOQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323073236 (397h51m49.093714572s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323756533 (397h51m49.094397868s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.326098634 (397h51m49.096739969s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.329541540 (397h51m49.100182875s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:22.918099454 (397h51m57.688740789s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:03:11.197265107 (397h52m45.967906442s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj, address:45.91.95.2:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197291919 (397h52m45.967933254s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12Yb4nEC9PfMTDs4iPU4WmSCHJrALQSDzKSwyD2z5g6WxM5Q5cB, address:45.91.95.3:4009): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197383253 (397h52m45.968024588s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection Jul 9 23:03:11.197407600 (397h52m45.968048935s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12X8Et5FNyAsE7ifEckfQemZkfnvYUuZ2Rn2xdxNxhmCmuDST2M, address:45.91.95.5:4022): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197423410 (397h52m45.968064745s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY, address:45.91.95.4:4001): protocol: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection Jul 9 23:03:11.197480189 (397h52m45.968121524s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197527659 (397h52m45.968168994s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197541516 (397h52m45.968182841s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: expected piece hash; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; EOF Jul 9 23:03:11.197562597 (397h52m45.968203932s) [seg_pos={0 0}, upload=318314]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 9 23:03:11.438173501 (397h52m46.208814836s) [upload=318314]: Done uploading segments... Jul 9 23:03:11.438197957 (397h52m46.208839292s) [upload=318314]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 21.976999186s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X...............X...X....................] duration: 12.778693287s FAIL (2025-07-09 23:05:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.778714787s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [......X...............................X..X.......X.......X..] duration: 40.385474639s FAIL (2025-07-09 23:04:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 0 (397h51m47.070257915s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.299863513 (397h51m47.070504848s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300155401 (397h51m47.070796736s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300500541 (397h51m47.071141886s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300707928 (397h51m47.071349263s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300895386 (397h51m47.071536712s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301093135 (397h51m47.07173447s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301289260 (397h51m47.071930595s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301483822 (397h51m47.072125147s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301668996 (397h51m47.072310332s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301881594 (397h51m47.072522929s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302036139 (397h51m47.072677484s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302541906 (397h51m47.073183241s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302904690 (397h51m47.073546025s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306087348 (397h51m47.076728683s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306494538 (397h51m47.077135883s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306742372 (397h51m47.077383707s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:13.699557207 (397h51m48.470198542s) [upload=318314]: Next returned nil segment Jul 9 23:02:13.699575412 (397h51m48.470216747s) [upload=318314]: Waiting for error group managing segments... Jul 9 23:02:14.004198502 (397h51m48.774839857s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.317361159 (397h51m49.088002494s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.318855398 (397h51m49.089496723s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.321324141 (397h51m49.091965476s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.322031012 (397h51m49.092672357s) [noise=true , address=77.90.31.12:4602 , piece_id=YALPUB6SRLSX54PKVC4UOKREEAXYZ22JE3FDX4OS3I6RUC7B5KOQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323073236 (397h51m49.093714572s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323756533 (397h51m49.094397868s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.326098634 (397h51m49.096739969s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.329541540 (397h51m49.100182875s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:22.918099454 (397h51m57.688740789s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:03:11.197265107 (397h52m45.967906442s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj, address:45.91.95.2:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197291919 (397h52m45.967933254s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12Yb4nEC9PfMTDs4iPU4WmSCHJrALQSDzKSwyD2z5g6WxM5Q5cB, address:45.91.95.3:4009): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197383253 (397h52m45.968024588s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection Jul 9 23:03:11.197407600 (397h52m45.968048935s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12X8Et5FNyAsE7ifEckfQemZkfnvYUuZ2Rn2xdxNxhmCmuDST2M, address:45.91.95.5:4022): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197423410 (397h52m45.968064745s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY, address:45.91.95.4:4001): protocol: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection Jul 9 23:03:11.197480189 (397h52m45.968121524s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197527659 (397h52m45.968168994s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197541516 (397h52m45.968182841s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: expected piece hash; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; EOF Jul 9 23:03:11.197562597 (397h52m45.968203932s) [seg_pos={0 0}, upload=318314]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 9 23:03:11.438173501 (397h52m46.208814836s) [upload=318314]: Done uploading segments... Jul 9 23:03:11.438197957 (397h52m46.208839292s) [upload=318314]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 40.385500549s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 15.696038468s FAIL (2025-07-09 23:04:36 UTC) orbiter warning: context deadline exceeded Duration: 15.69605982s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X...............X...X...................] duration: 5.93664972s FAIL (2025-07-09 23:03:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.936671644s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.......X...............................X..X.......X.......X.] duration: 25.397857209s FAIL (2025-07-09 23:03:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 0 (397h51m47.070257915s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.299863513 (397h51m47.070504848s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300155401 (397h51m47.070796736s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300500541 (397h51m47.071141886s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300707928 (397h51m47.071349263s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300895386 (397h51m47.071536712s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301093135 (397h51m47.07173447s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301289260 (397h51m47.071930595s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301483822 (397h51m47.072125147s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301668996 (397h51m47.072310332s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301881594 (397h51m47.072522929s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302036139 (397h51m47.072677484s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302541906 (397h51m47.073183241s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302904690 (397h51m47.073546025s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306087348 (397h51m47.076728683s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306494538 (397h51m47.077135883s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306742372 (397h51m47.077383707s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:13.699557207 (397h51m48.470198542s) [upload=318314]: Next returned nil segment Jul 9 23:02:13.699575412 (397h51m48.470216747s) [upload=318314]: Waiting for error group managing segments... Jul 9 23:02:14.004198502 (397h51m48.774839857s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.317361159 (397h51m49.088002494s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.318855398 (397h51m49.089496723s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.321324141 (397h51m49.091965476s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.322031012 (397h51m49.092672357s) [noise=true , address=77.90.31.12:4602 , piece_id=YALPUB6SRLSX54PKVC4UOKREEAXYZ22JE3FDX4OS3I6RUC7B5KOQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323073236 (397h51m49.093714572s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323756533 (397h51m49.094397868s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.326098634 (397h51m49.096739969s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.329541540 (397h51m49.100182875s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:22.918099454 (397h51m57.688740789s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:03:11.197265107 (397h52m45.967906442s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj, address:45.91.95.2:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197291919 (397h52m45.967933254s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12Yb4nEC9PfMTDs4iPU4WmSCHJrALQSDzKSwyD2z5g6WxM5Q5cB, address:45.91.95.3:4009): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197383253 (397h52m45.968024588s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection Jul 9 23:03:11.197407600 (397h52m45.968048935s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12X8Et5FNyAsE7ifEckfQemZkfnvYUuZ2Rn2xdxNxhmCmuDST2M, address:45.91.95.5:4022): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197423410 (397h52m45.968064745s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY, address:45.91.95.4:4001): protocol: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection Jul 9 23:03:11.197480189 (397h52m45.968121524s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197527659 (397h52m45.968168994s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197541516 (397h52m45.968182841s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: expected piece hash; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; EOF Jul 9 23:03:11.197562597 (397h52m45.968203932s) [seg_pos={0 0}, upload=318314]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 9 23:03:11.438173501 (397h52m46.208814836s) [upload=318314]: Done uploading segments... Jul 9 23:03:11.438197957 (397h52m46.208839292s) [upload=318314]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 25.397874412s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 22.147841344s FAIL (2025-07-09 23:03:31 UTC) orbiter warning: context deadline exceeded Duration: 22.147868786s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X...............X...X..................] duration: 6.053465622s FAIL (2025-07-09 23:03:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.053504145s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [........X...............................X..X.......X.......X] duration: 1m0.241956877s FAIL (2025-07-09 23:03:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 0 (397h51m47.070257915s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.299863513 (397h51m47.070504848s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300155401 (397h51m47.070796736s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300500541 (397h51m47.071141886s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300707928 (397h51m47.071349263s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.300895386 (397h51m47.071536712s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301093135 (397h51m47.07173447s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301289260 (397h51m47.071930595s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301483822 (397h51m47.072125147s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301668996 (397h51m47.072310332s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.301881594 (397h51m47.072522929s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302036139 (397h51m47.072677484s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302541906 (397h51m47.073183241s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.302904690 (397h51m47.073546025s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306087348 (397h51m47.076728683s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306494538 (397h51m47.077135883s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:12.306742372 (397h51m47.077383707s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Uploading piece... Jul 9 23:02:13.699557207 (397h51m48.470198542s) [upload=318314]: Next returned nil segment Jul 9 23:02:13.699575412 (397h51m48.470216747s) [upload=318314]: Waiting for error group managing segments... Jul 9 23:02:14.004198502 (397h51m48.774839857s) [noise=true , address=79.127.220.101:4071 , piece_id=PCSVC42EYQVKAAHWVZOA4QNW3GRGOZINTK3A37OUKMEM37L2TURA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.317361159 (397h51m49.088002494s) [noise=true , address=77.90.31.17:4802 , piece_id=OL6BCBP4YTNGDX23VQXEU7MBPWG42BYZ3AZYJ5DOWMOPYIZ7HKJA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.318855398 (397h51m49.089496723s) [noise=true , address=77.90.31.14:4688 , piece_id=BH5GKPDTKYKS7LMOSBFGCAK3GPTMOHS2XVHPBQ3IIXYUR5KOMNQQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.321324141 (397h51m49.091965476s) [noise=true , address=77.90.31.11:4575 , piece_id=QKIXONTL5DHGG2U6FCE4G5S2MGMLF6M342LSA5PIUEYAFRILAVEA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.322031012 (397h51m49.092672357s) [noise=true , address=77.90.31.12:4602 , piece_id=YALPUB6SRLSX54PKVC4UOKREEAXYZ22JE3FDX4OS3I6RUC7B5KOQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323073236 (397h51m49.093714572s) [noise=true , address=77.90.31.21:4884 , piece_id=Y6KXVSWCMBNCUKEXVDYT5MJTSHXQ6BTWQMQ755BPYXJZ54F6HPZA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.323756533 (397h51m49.094397868s) [noise=true , address=77.90.31.13:4645 , piece_id=SCDGEUTMKVRVHG4YA7DC6OYBRBEQQ7AJ3TJUGS3OXBAQ6E6VUZMQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.326098634 (397h51m49.096739969s) [noise=true , address=77.90.31.16:4738 , piece_id=EOD3ITGEW6B5TKWAAX5OLRSBHWLUSZ3HP6WRRC5JRNYOBURXJLYA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:14.329541540 (397h51m49.100182875s) [noise=true , address=77.90.31.19:4876 , piece_id=MFCUKJNLJOGD6BSOJX34NF6ETPQB6GKHKHZZOBE27BIFS5C5QQ4A, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:02:22.918099454 (397h51m57.688740789s) [noise=true , address=79.127.219.38:4031 , piece_id=CPSX3N3IDEWKUMHBQ6H6WMBFWDG3TGBOFV4AF464NRE4BTMRWYKQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: <nil> Jul 9 23:03:11.197265107 (397h52m45.967906442s) [noise=true , address=45.91.95.2:4015 , piece_id=SSMTWH5WQCMLFZOXRZX4WGCQDSHMUTD2W555XO6AA7OJFBUV72YA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj, address:45.91.95.2:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197291919 (397h52m45.967933254s) [noise=true , address=45.91.95.3:4009 , piece_id=M6DJJMELQOAIEXU2OY2BUSGY63LI6WY6SFHVG6DFO64RS63J3SQA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12Yb4nEC9PfMTDs4iPU4WmSCHJrALQSDzKSwyD2z5g6WxM5Q5cB, address:45.91.95.3:4009): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197383253 (397h52m45.968024588s) [noise=true , address=104.36.82.23:4447 , piece_id=QJ7UQG24TH5QAPDOSOCMCI7B4S5V7ZCLDTIBQ7T5I5IWOLINDZWQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:104.36.82.23:4447): protocol: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48822->104.36.82.23:4447: use of closed network connection Jul 9 23:03:11.197407600 (397h52m45.968048935s) [noise=true , address=45.91.95.5:4022 , piece_id=63X6LG462THOR7RUHRZ4MGYRZDJZUD3CFUIE57GY4ENGMNR72GGA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12X8Et5FNyAsE7ifEckfQemZkfnvYUuZ2Rn2xdxNxhmCmuDST2M, address:45.91.95.5:4022): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197423410 (397h52m45.968064745s) [noise=true , address=45.91.95.4:4001 , piece_id=DWNFECN6K3SEVJGBBKAF3QGK5TG4NMFGIZSKDB2NPV5APFBIEYRA, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY, address:45.91.95.4:4001): protocol: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33470->45.91.95.4:4001: use of closed network connection Jul 9 23:03:11.197480189 (397h52m45.968121524s) [noise=true , address=104.36.82.36:4624 , piece_id=3MJAMTZ3D5HURGA37NIOBVLFCYUDNGK3X42KAJ2I6RG2DB5Q3YSQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197527659 (397h52m45.968168994s) [noise=true , address=104.36.82.44:4899 , piece_id=BJ6QCUENRVC2MLUQRAL5NL7KP6OELTJWZ2NBYZJ3HPIJH3EGIKFQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12b4pZVQRnjcQNn45VTgdsz5fpjYPMAzSXGR5KJ5BP9XHMkRnRw, address:104.36.82.44:4899): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 23:03:11.197541516 (397h52m45.968182841s) [noise=true , address=104.36.82.19:4410 , piece_id=GIE3YQ3NHF44OHPJXNZQCCQLAYMH7WVHU5RYCC6UU3A5L4MNBDVQ, seg_pos={0 0}, upload=318314]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: expected piece hash; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; write tcp 188.166.225.170:35686->104.36.82.19:4410: use of closed network connection; EOF Jul 9 23:03:11.197562597 (397h52m45.968203932s) [seg_pos={0 0}, upload=318314]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 9 23:03:11.438173501 (397h52m46.208814836s) [upload=318314]: Done uploading segments... Jul 9 23:03:11.438197957 (397h52m46.208839292s) [upload=318314]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.241969693s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X...............X...X.................] duration: 5.928133919s FAIL (2025-07-09 23:03:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.928159788s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 26.886558634s FAIL (2025-07-09 23:02:39 UTC) orbiter warning: context deadline exceeded Duration: 26.886571348s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X...............X...X................] duration: 6.016283829s FAIL (2025-07-09 23:02:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.016307194s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 17.627909238s FAIL (2025-07-09 23:02:12 UTC) orbiter warning: context deadline exceeded Duration: 17.627927472s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.........X...............................X..X.......X.......] duration: 22.560557914s FAIL (2025-07-09 23:02:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:56:18.858763654 (397h45m53.629404999s) [seg_pos={0 0}, upload=318232]: Begin upload segment... Jul 9 22:56:18.858823458 (397h45m53.629464803s) [seg_pos={0 0}, upload=318232]: Done begin upload segment. Jul 9 22:56:18.858851111 (397h45m53.629492446s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.858870588 (397h45m53.629511933s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859212432 (397h45m53.629853767s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859462049 (397h45m53.630103384s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859709823 (397h45m53.630351159s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859904867 (397h45m53.630546202s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859980301 (397h45m53.630621636s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860265426 (397h45m53.630906761s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860290635 (397h45m53.63093196s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860570911 (397h45m53.631212246s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860943713 (397h45m53.631585058s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861191057 (397h45m53.631832392s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861344721 (397h45m53.631986056s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861414003 (397h45m53.632055338s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861634194 (397h45m53.63227556s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861762630 (397h45m53.632403956s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862277024 (397h45m53.632918359s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862683442 (397h45m53.633324777s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:20.347101698 (397h45m55.117743033s) [upload=318232]: Next returned nil segment Jul 9 22:56:20.347116396 (397h45m55.117757731s) [upload=318232]: Waiting for error group managing segments... Jul 9 22:56:20.861836827 (397h45m55.632478223s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.862480549 (397h45m55.633121884s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.863211167 (397h45m55.633852492s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871004916 (397h45m55.641646251s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871950425 (397h45m55.64259176s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.880814612 (397h45m55.651455947s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.884716216 (397h45m55.655357542s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:24.950988074 (397h45m59.721629409s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:25.194251943 (397h45m59.964893278s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.103374269 (397h46m8.874015604s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.436342364 (397h46m9.206983699s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:05.601199152 (397h46m40.371840487s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:17.760765568 (397h46m52.531406903s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1XWYrjZ3FuoB117j5iRB2zxgeA3We8LHaUUHW6B8uiFkD88Ad7, address:45.91.95.2:4010): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760821976 (397h46m52.531463311s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1GsGdvhc9tqAEYWDLDVG9Wrszda4EjYR9eTbFgepZNF2dEFazy, address:45.91.95.2:4023): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760947976 (397h46m52.531589301s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:12kFYkJi2nr3EVpxZ6NVWcQwX8hRy3fXkfVmBNXjcmyPK2EoHMt, address:45.91.95.4:4012): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761017459 (397h46m52.531658794s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1zEQhyuVxSHwdDUgDfcTog2W3oLBx4z9bYUMHpaaoLU1BTU9DQ, address:104.36.82.50:4027): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761104556 (397h46m52.531745891s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1bupgxDMcVUwTNB2uUHNeiG7pGira7jssXK1byhozFu2BA4ma1, address:45.91.95.3:4031): protocol: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection Jul 9 22:57:17.761152347 (397h46m52.531793672s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:126eg5nWqSdmnm9UUUpwEidVVFe9cgaNRraS2efAvtJcwg8qXfq, address:45.91.95.5:4030): protocol: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection Jul 9 22:57:17.761167226 (397h46m52.531808561s) [seg_pos={0 0}, upload=318232]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 9 22:57:17.992237959 (397h46m52.762879294s) [upload=318232]: Done uploading segments... Jul 9 22:57:17.992261995 (397h46m52.76290332s) [upload=318232]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 22.560582189s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 23.776938678s FAIL (2025-07-09 23:01:43 UTC) orbiter warning: context deadline exceeded Duration: 23.776962173s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X...............X...X...............] duration: 6.353270203s FAIL (2025-07-09 23:01:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.353285393s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [X.........X...............................X..X.......X......] duration: 16.260687607s FAIL (2025-07-09 23:01:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:56:18.858763654 (397h45m53.629404999s) [seg_pos={0 0}, upload=318232]: Begin upload segment... Jul 9 22:56:18.858823458 (397h45m53.629464803s) [seg_pos={0 0}, upload=318232]: Done begin upload segment. Jul 9 22:56:18.858851111 (397h45m53.629492446s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.858870588 (397h45m53.629511933s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859212432 (397h45m53.629853767s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859462049 (397h45m53.630103384s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859709823 (397h45m53.630351159s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859904867 (397h45m53.630546202s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859980301 (397h45m53.630621636s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860265426 (397h45m53.630906761s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860290635 (397h45m53.63093196s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860570911 (397h45m53.631212246s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860943713 (397h45m53.631585058s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861191057 (397h45m53.631832392s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861344721 (397h45m53.631986056s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861414003 (397h45m53.632055338s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861634194 (397h45m53.63227556s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861762630 (397h45m53.632403956s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862277024 (397h45m53.632918359s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862683442 (397h45m53.633324777s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:20.347101698 (397h45m55.117743033s) [upload=318232]: Next returned nil segment Jul 9 22:56:20.347116396 (397h45m55.117757731s) [upload=318232]: Waiting for error group managing segments... Jul 9 22:56:20.861836827 (397h45m55.632478223s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.862480549 (397h45m55.633121884s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.863211167 (397h45m55.633852492s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871004916 (397h45m55.641646251s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871950425 (397h45m55.64259176s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.880814612 (397h45m55.651455947s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.884716216 (397h45m55.655357542s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:24.950988074 (397h45m59.721629409s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:25.194251943 (397h45m59.964893278s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.103374269 (397h46m8.874015604s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.436342364 (397h46m9.206983699s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:05.601199152 (397h46m40.371840487s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:17.760765568 (397h46m52.531406903s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1XWYrjZ3FuoB117j5iRB2zxgeA3We8LHaUUHW6B8uiFkD88Ad7, address:45.91.95.2:4010): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760821976 (397h46m52.531463311s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1GsGdvhc9tqAEYWDLDVG9Wrszda4EjYR9eTbFgepZNF2dEFazy, address:45.91.95.2:4023): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760947976 (397h46m52.531589301s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:12kFYkJi2nr3EVpxZ6NVWcQwX8hRy3fXkfVmBNXjcmyPK2EoHMt, address:45.91.95.4:4012): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761017459 (397h46m52.531658794s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1zEQhyuVxSHwdDUgDfcTog2W3oLBx4z9bYUMHpaaoLU1BTU9DQ, address:104.36.82.50:4027): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761104556 (397h46m52.531745891s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1bupgxDMcVUwTNB2uUHNeiG7pGira7jssXK1byhozFu2BA4ma1, address:45.91.95.3:4031): protocol: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection Jul 9 22:57:17.761152347 (397h46m52.531793672s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:126eg5nWqSdmnm9UUUpwEidVVFe9cgaNRraS2efAvtJcwg8qXfq, address:45.91.95.5:4030): protocol: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection Jul 9 22:57:17.761167226 (397h46m52.531808561s) [seg_pos={0 0}, upload=318232]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 9 22:57:17.992237959 (397h46m52.762879294s) [upload=318232]: Done uploading segments... Jul 9 22:57:17.992261995 (397h46m52.76290332s) [upload=318232]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 16.260704199s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X...............X...X..............] duration: 11.957614498s FAIL (2025-07-09 23:01:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.957637883s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.X.........X...............................X..X.......X.....] duration: 59.895143967s FAIL (2025-07-09 23:01:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:56:18.858763654 (397h45m53.629404999s) [seg_pos={0 0}, upload=318232]: Begin upload segment... Jul 9 22:56:18.858823458 (397h45m53.629464803s) [seg_pos={0 0}, upload=318232]: Done begin upload segment. Jul 9 22:56:18.858851111 (397h45m53.629492446s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.858870588 (397h45m53.629511933s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859212432 (397h45m53.629853767s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859462049 (397h45m53.630103384s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859709823 (397h45m53.630351159s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859904867 (397h45m53.630546202s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859980301 (397h45m53.630621636s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860265426 (397h45m53.630906761s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860290635 (397h45m53.63093196s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860570911 (397h45m53.631212246s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860943713 (397h45m53.631585058s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861191057 (397h45m53.631832392s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861344721 (397h45m53.631986056s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861414003 (397h45m53.632055338s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861634194 (397h45m53.63227556s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861762630 (397h45m53.632403956s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862277024 (397h45m53.632918359s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862683442 (397h45m53.633324777s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:20.347101698 (397h45m55.117743033s) [upload=318232]: Next returned nil segment Jul 9 22:56:20.347116396 (397h45m55.117757731s) [upload=318232]: Waiting for error group managing segments... Jul 9 22:56:20.861836827 (397h45m55.632478223s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.862480549 (397h45m55.633121884s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.863211167 (397h45m55.633852492s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871004916 (397h45m55.641646251s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871950425 (397h45m55.64259176s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.880814612 (397h45m55.651455947s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.884716216 (397h45m55.655357542s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:24.950988074 (397h45m59.721629409s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:25.194251943 (397h45m59.964893278s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.103374269 (397h46m8.874015604s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.436342364 (397h46m9.206983699s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:05.601199152 (397h46m40.371840487s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:17.760765568 (397h46m52.531406903s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1XWYrjZ3FuoB117j5iRB2zxgeA3We8LHaUUHW6B8uiFkD88Ad7, address:45.91.95.2:4010): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760821976 (397h46m52.531463311s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1GsGdvhc9tqAEYWDLDVG9Wrszda4EjYR9eTbFgepZNF2dEFazy, address:45.91.95.2:4023): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760947976 (397h46m52.531589301s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:12kFYkJi2nr3EVpxZ6NVWcQwX8hRy3fXkfVmBNXjcmyPK2EoHMt, address:45.91.95.4:4012): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761017459 (397h46m52.531658794s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1zEQhyuVxSHwdDUgDfcTog2W3oLBx4z9bYUMHpaaoLU1BTU9DQ, address:104.36.82.50:4027): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761104556 (397h46m52.531745891s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1bupgxDMcVUwTNB2uUHNeiG7pGira7jssXK1byhozFu2BA4ma1, address:45.91.95.3:4031): protocol: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection Jul 9 22:57:17.761152347 (397h46m52.531793672s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:126eg5nWqSdmnm9UUUpwEidVVFe9cgaNRraS2efAvtJcwg8qXfq, address:45.91.95.5:4030): protocol: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection Jul 9 22:57:17.761167226 (397h46m52.531808561s) [seg_pos={0 0}, upload=318232]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 9 22:57:17.992237959 (397h46m52.762879294s) [upload=318232]: Done uploading segments... Jul 9 22:57:17.992261995 (397h46m52.76290332s) [upload=318232]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 59.895169376s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 21.913427839s FAIL (2025-07-09 23:00:43 UTC) orbiter warning: context deadline exceeded Duration: 21.913454209s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.........................X...............X...X.............] duration: 6.021568168s FAIL (2025-07-09 23:00:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.021594168s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [..X.........X...............................X..X.......X....] duration: 52.564825522s FAIL (2025-07-09 23:00:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:56:18.858763654 (397h45m53.629404999s) [seg_pos={0 0}, upload=318232]: Begin upload segment... Jul 9 22:56:18.858823458 (397h45m53.629464803s) [seg_pos={0 0}, upload=318232]: Done begin upload segment. Jul 9 22:56:18.858851111 (397h45m53.629492446s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.858870588 (397h45m53.629511933s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859212432 (397h45m53.629853767s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859462049 (397h45m53.630103384s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859709823 (397h45m53.630351159s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859904867 (397h45m53.630546202s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859980301 (397h45m53.630621636s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860265426 (397h45m53.630906761s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860290635 (397h45m53.63093196s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860570911 (397h45m53.631212246s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860943713 (397h45m53.631585058s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861191057 (397h45m53.631832392s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861344721 (397h45m53.631986056s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861414003 (397h45m53.632055338s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861634194 (397h45m53.63227556s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861762630 (397h45m53.632403956s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862277024 (397h45m53.632918359s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862683442 (397h45m53.633324777s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:20.347101698 (397h45m55.117743033s) [upload=318232]: Next returned nil segment Jul 9 22:56:20.347116396 (397h45m55.117757731s) [upload=318232]: Waiting for error group managing segments... Jul 9 22:56:20.861836827 (397h45m55.632478223s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.862480549 (397h45m55.633121884s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.863211167 (397h45m55.633852492s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871004916 (397h45m55.641646251s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871950425 (397h45m55.64259176s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.880814612 (397h45m55.651455947s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.884716216 (397h45m55.655357542s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:24.950988074 (397h45m59.721629409s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:25.194251943 (397h45m59.964893278s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.103374269 (397h46m8.874015604s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.436342364 (397h46m9.206983699s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:05.601199152 (397h46m40.371840487s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:17.760765568 (397h46m52.531406903s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1XWYrjZ3FuoB117j5iRB2zxgeA3We8LHaUUHW6B8uiFkD88Ad7, address:45.91.95.2:4010): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760821976 (397h46m52.531463311s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1GsGdvhc9tqAEYWDLDVG9Wrszda4EjYR9eTbFgepZNF2dEFazy, address:45.91.95.2:4023): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760947976 (397h46m52.531589301s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:12kFYkJi2nr3EVpxZ6NVWcQwX8hRy3fXkfVmBNXjcmyPK2EoHMt, address:45.91.95.4:4012): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761017459 (397h46m52.531658794s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1zEQhyuVxSHwdDUgDfcTog2W3oLBx4z9bYUMHpaaoLU1BTU9DQ, address:104.36.82.50:4027): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761104556 (397h46m52.531745891s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1bupgxDMcVUwTNB2uUHNeiG7pGira7jssXK1byhozFu2BA4ma1, address:45.91.95.3:4031): protocol: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection Jul 9 22:57:17.761152347 (397h46m52.531793672s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:126eg5nWqSdmnm9UUUpwEidVVFe9cgaNRraS2efAvtJcwg8qXfq, address:45.91.95.5:4030): protocol: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection Jul 9 22:57:17.761167226 (397h46m52.531808561s) [seg_pos={0 0}, upload=318232]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 9 22:57:17.992237959 (397h46m52.762879294s) [upload=318232]: Done uploading segments... Jul 9 22:57:17.992261995 (397h46m52.76290332s) [upload=318232]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 52.564850521s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 28.226758337s FAIL (2025-07-09 23:00:04 UTC) orbiter warning: context deadline exceeded Duration: 28.226784658s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.........................X...............X...X............] duration: 6.208130667s FAIL (2025-07-09 22:59:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.208152899s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [...X.........X...............................X..X.......X...] duration: 20.815686842s FAIL (2025-07-09 22:59:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:56:18.858763654 (397h45m53.629404999s) [seg_pos={0 0}, upload=318232]: Begin upload segment... Jul 9 22:56:18.858823458 (397h45m53.629464803s) [seg_pos={0 0}, upload=318232]: Done begin upload segment. Jul 9 22:56:18.858851111 (397h45m53.629492446s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.858870588 (397h45m53.629511933s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859212432 (397h45m53.629853767s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859462049 (397h45m53.630103384s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859709823 (397h45m53.630351159s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859904867 (397h45m53.630546202s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859980301 (397h45m53.630621636s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860265426 (397h45m53.630906761s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860290635 (397h45m53.63093196s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860570911 (397h45m53.631212246s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860943713 (397h45m53.631585058s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861191057 (397h45m53.631832392s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861344721 (397h45m53.631986056s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861414003 (397h45m53.632055338s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861634194 (397h45m53.63227556s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861762630 (397h45m53.632403956s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862277024 (397h45m53.632918359s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862683442 (397h45m53.633324777s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:20.347101698 (397h45m55.117743033s) [upload=318232]: Next returned nil segment Jul 9 22:56:20.347116396 (397h45m55.117757731s) [upload=318232]: Waiting for error group managing segments... Jul 9 22:56:20.861836827 (397h45m55.632478223s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.862480549 (397h45m55.633121884s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.863211167 (397h45m55.633852492s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871004916 (397h45m55.641646251s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871950425 (397h45m55.64259176s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.880814612 (397h45m55.651455947s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.884716216 (397h45m55.655357542s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:24.950988074 (397h45m59.721629409s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:25.194251943 (397h45m59.964893278s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.103374269 (397h46m8.874015604s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.436342364 (397h46m9.206983699s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:05.601199152 (397h46m40.371840487s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:17.760765568 (397h46m52.531406903s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1XWYrjZ3FuoB117j5iRB2zxgeA3We8LHaUUHW6B8uiFkD88Ad7, address:45.91.95.2:4010): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760821976 (397h46m52.531463311s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1GsGdvhc9tqAEYWDLDVG9Wrszda4EjYR9eTbFgepZNF2dEFazy, address:45.91.95.2:4023): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760947976 (397h46m52.531589301s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:12kFYkJi2nr3EVpxZ6NVWcQwX8hRy3fXkfVmBNXjcmyPK2EoHMt, address:45.91.95.4:4012): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761017459 (397h46m52.531658794s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1zEQhyuVxSHwdDUgDfcTog2W3oLBx4z9bYUMHpaaoLU1BTU9DQ, address:104.36.82.50:4027): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761104556 (397h46m52.531745891s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1bupgxDMcVUwTNB2uUHNeiG7pGira7jssXK1byhozFu2BA4ma1, address:45.91.95.3:4031): protocol: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection Jul 9 22:57:17.761152347 (397h46m52.531793672s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:126eg5nWqSdmnm9UUUpwEidVVFe9cgaNRraS2efAvtJcwg8qXfq, address:45.91.95.5:4030): protocol: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection Jul 9 22:57:17.761167226 (397h46m52.531808561s) [seg_pos={0 0}, upload=318232]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 9 22:57:17.992237959 (397h46m52.762879294s) [upload=318232]: Done uploading segments... Jul 9 22:57:17.992261995 (397h46m52.76290332s) [upload=318232]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 20.815703845s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 21.021930691s FAIL (2025-07-09 22:58:54 UTC) orbiter warning: context deadline exceeded Duration: 21.021947443s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [....X.........X...............................X..X.......X..] duration: 36.229301949s FAIL (2025-07-09 22:58:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:56:18.858763654 (397h45m53.629404999s) [seg_pos={0 0}, upload=318232]: Begin upload segment... Jul 9 22:56:18.858823458 (397h45m53.629464803s) [seg_pos={0 0}, upload=318232]: Done begin upload segment. Jul 9 22:56:18.858851111 (397h45m53.629492446s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.858870588 (397h45m53.629511933s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859212432 (397h45m53.629853767s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859462049 (397h45m53.630103384s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859709823 (397h45m53.630351159s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859904867 (397h45m53.630546202s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859980301 (397h45m53.630621636s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860265426 (397h45m53.630906761s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860290635 (397h45m53.63093196s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860570911 (397h45m53.631212246s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860943713 (397h45m53.631585058s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861191057 (397h45m53.631832392s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861344721 (397h45m53.631986056s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861414003 (397h45m53.632055338s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861634194 (397h45m53.63227556s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861762630 (397h45m53.632403956s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862277024 (397h45m53.632918359s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862683442 (397h45m53.633324777s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:20.347101698 (397h45m55.117743033s) [upload=318232]: Next returned nil segment Jul 9 22:56:20.347116396 (397h45m55.117757731s) [upload=318232]: Waiting for error group managing segments... Jul 9 22:56:20.861836827 (397h45m55.632478223s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.862480549 (397h45m55.633121884s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.863211167 (397h45m55.633852492s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871004916 (397h45m55.641646251s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871950425 (397h45m55.64259176s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.880814612 (397h45m55.651455947s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.884716216 (397h45m55.655357542s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:24.950988074 (397h45m59.721629409s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:25.194251943 (397h45m59.964893278s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.103374269 (397h46m8.874015604s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.436342364 (397h46m9.206983699s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:05.601199152 (397h46m40.371840487s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:17.760765568 (397h46m52.531406903s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1XWYrjZ3FuoB117j5iRB2zxgeA3We8LHaUUHW6B8uiFkD88Ad7, address:45.91.95.2:4010): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760821976 (397h46m52.531463311s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1GsGdvhc9tqAEYWDLDVG9Wrszda4EjYR9eTbFgepZNF2dEFazy, address:45.91.95.2:4023): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760947976 (397h46m52.531589301s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:12kFYkJi2nr3EVpxZ6NVWcQwX8hRy3fXkfVmBNXjcmyPK2EoHMt, address:45.91.95.4:4012): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761017459 (397h46m52.531658794s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1zEQhyuVxSHwdDUgDfcTog2W3oLBx4z9bYUMHpaaoLU1BTU9DQ, address:104.36.82.50:4027): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761104556 (397h46m52.531745891s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1bupgxDMcVUwTNB2uUHNeiG7pGira7jssXK1byhozFu2BA4ma1, address:45.91.95.3:4031): protocol: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection Jul 9 22:57:17.761152347 (397h46m52.531793672s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:126eg5nWqSdmnm9UUUpwEidVVFe9cgaNRraS2efAvtJcwg8qXfq, address:45.91.95.5:4030): protocol: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection Jul 9 22:57:17.761167226 (397h46m52.531808561s) [seg_pos={0 0}, upload=318232]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 9 22:57:17.992237959 (397h46m52.762879294s) [upload=318232]: Done uploading segments... Jul 9 22:57:17.992261995 (397h46m52.76290332s) [upload=318232]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 36.229334392s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.........................X...............X...X...........] duration: 6.072939046s FAIL (2025-07-09 22:58:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.072961499s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X.....................X.....................................] duration: 29.398166021s FAIL (2025-07-09 22:58:07 UTC) orbiter warning: context deadline exceeded Duration: 29.398193562s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.........................X...............X...X..........] duration: 5.912878344s FAIL (2025-07-09 22:57:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.912914332s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.....X.........X...............................X..X.......X.] duration: 34.226436916s FAIL (2025-07-09 22:57:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:56:18.858763654 (397h45m53.629404999s) [seg_pos={0 0}, upload=318232]: Begin upload segment... Jul 9 22:56:18.858823458 (397h45m53.629464803s) [seg_pos={0 0}, upload=318232]: Done begin upload segment. Jul 9 22:56:18.858851111 (397h45m53.629492446s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.858870588 (397h45m53.629511933s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859212432 (397h45m53.629853767s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859462049 (397h45m53.630103384s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859709823 (397h45m53.630351159s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859904867 (397h45m53.630546202s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859980301 (397h45m53.630621636s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860265426 (397h45m53.630906761s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860290635 (397h45m53.63093196s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860570911 (397h45m53.631212246s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860943713 (397h45m53.631585058s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861191057 (397h45m53.631832392s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861344721 (397h45m53.631986056s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861414003 (397h45m53.632055338s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861634194 (397h45m53.63227556s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861762630 (397h45m53.632403956s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862277024 (397h45m53.632918359s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862683442 (397h45m53.633324777s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:20.347101698 (397h45m55.117743033s) [upload=318232]: Next returned nil segment Jul 9 22:56:20.347116396 (397h45m55.117757731s) [upload=318232]: Waiting for error group managing segments... Jul 9 22:56:20.861836827 (397h45m55.632478223s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.862480549 (397h45m55.633121884s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.863211167 (397h45m55.633852492s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871004916 (397h45m55.641646251s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871950425 (397h45m55.64259176s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.880814612 (397h45m55.651455947s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.884716216 (397h45m55.655357542s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:24.950988074 (397h45m59.721629409s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:25.194251943 (397h45m59.964893278s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.103374269 (397h46m8.874015604s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.436342364 (397h46m9.206983699s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:05.601199152 (397h46m40.371840487s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:17.760765568 (397h46m52.531406903s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1XWYrjZ3FuoB117j5iRB2zxgeA3We8LHaUUHW6B8uiFkD88Ad7, address:45.91.95.2:4010): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760821976 (397h46m52.531463311s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1GsGdvhc9tqAEYWDLDVG9Wrszda4EjYR9eTbFgepZNF2dEFazy, address:45.91.95.2:4023): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760947976 (397h46m52.531589301s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:12kFYkJi2nr3EVpxZ6NVWcQwX8hRy3fXkfVmBNXjcmyPK2EoHMt, address:45.91.95.4:4012): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761017459 (397h46m52.531658794s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1zEQhyuVxSHwdDUgDfcTog2W3oLBx4z9bYUMHpaaoLU1BTU9DQ, address:104.36.82.50:4027): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761104556 (397h46m52.531745891s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1bupgxDMcVUwTNB2uUHNeiG7pGira7jssXK1byhozFu2BA4ma1, address:45.91.95.3:4031): protocol: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection Jul 9 22:57:17.761152347 (397h46m52.531793672s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:126eg5nWqSdmnm9UUUpwEidVVFe9cgaNRraS2efAvtJcwg8qXfq, address:45.91.95.5:4030): protocol: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection Jul 9 22:57:17.761167226 (397h46m52.531808561s) [seg_pos={0 0}, upload=318232]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 9 22:57:17.992237959 (397h46m52.762879294s) [upload=318232]: Done uploading segments... Jul 9 22:57:17.992261995 (397h46m52.76290332s) [upload=318232]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 34.226451654s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X.....................X....................................] duration: 25.53957479s FAIL (2025-07-09 22:57:30 UTC) orbiter warning: context deadline exceeded Duration: 25.539607613s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [......X.........X...............................X..X.......X] duration: 1m0.232598224s FAIL (2025-07-09 22:57:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:56:18.858763654 (397h45m53.629404999s) [seg_pos={0 0}, upload=318232]: Begin upload segment... Jul 9 22:56:18.858823458 (397h45m53.629464803s) [seg_pos={0 0}, upload=318232]: Done begin upload segment. Jul 9 22:56:18.858851111 (397h45m53.629492446s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.858870588 (397h45m53.629511933s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859212432 (397h45m53.629853767s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859462049 (397h45m53.630103384s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859709823 (397h45m53.630351159s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859904867 (397h45m53.630546202s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.859980301 (397h45m53.630621636s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860265426 (397h45m53.630906761s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860290635 (397h45m53.63093196s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860570911 (397h45m53.631212246s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.860943713 (397h45m53.631585058s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861191057 (397h45m53.631832392s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861344721 (397h45m53.631986056s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861414003 (397h45m53.632055338s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861634194 (397h45m53.63227556s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.861762630 (397h45m53.632403956s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862277024 (397h45m53.632918359s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:18.862683442 (397h45m53.633324777s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Uploading piece... Jul 9 22:56:20.347101698 (397h45m55.117743033s) [upload=318232]: Next returned nil segment Jul 9 22:56:20.347116396 (397h45m55.117757731s) [upload=318232]: Waiting for error group managing segments... Jul 9 22:56:20.861836827 (397h45m55.632478223s) [noise=true , address=77.90.31.16:4759 , piece_id=LPDRHYNAPSZXTHHCC7NQ2FJIYZ5DFF5R2AMLX64LN3FY5MRVRUIA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.862480549 (397h45m55.633121884s) [noise=true , address=77.90.31.11:4569 , piece_id=JHJBWJFVLAB4XTEJRKCD22UJ7MM7NKJDA74FE55ZF5ZFF6PG4OBQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.863211167 (397h45m55.633852492s) [noise=true , address=77.90.31.15:4705 , piece_id=ZBDDSRKT4P6BCM3V6TB7WNFBI63JMLRBMW2QCPAVNFAHS34LRBOQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871004916 (397h45m55.641646251s) [noise=true , address=77.90.31.19:4858 , piece_id=LFRKKB3C4SLVANBLZ4VKUHR5E6KF54ZXOPKGADHVW573HMP32PMA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.871950425 (397h45m55.64259176s) [noise=true , address=77.90.31.18:4822 , piece_id=LUY7CGVIZJ3TXRQDWARUQDCMOQ6IB4QZRPPKTRTM5YB3LFWBWFBA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.880814612 (397h45m55.651455947s) [noise=true , address=77.90.31.10:4551 , piece_id=6IA46PLBYXT4W2VXLKTUC5SD5V7G5JGUVYRJPBVE3UEYKKJWGSCA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:20.884716216 (397h45m55.655357542s) [noise=true , address=77.90.31.21:4892 , piece_id=TZMVNICQWL4CGFRSUA6RA5ZG4IPLNIPJ5BIRPUNUTXGP42JTIDVA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:24.950988074 (397h45m59.721629409s) [noise=true , address=79.127.219.38:4019 , piece_id=L7H35Q3L7BTYZBKTS3ZN7Z6Q47OEQNL5SX26QHDYY4VH4FOSNLPQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:25.194251943 (397h45m59.964893278s) [noise=true , address=45.91.116.4:4955 , piece_id=SUW6GB4XW3K6XYQVXUOY6SW3PYURBP4LLYZN475HGJQUI6AOMVKA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.103374269 (397h46m8.874015604s) [noise=true , address=104.36.82.40:4971 , piece_id=W5DI2BBHUAAPC74PX7YZ4ZTW6EPN6TOZVMCKH72BGSHPZRAEWL4Q, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:56:34.436342364 (397h46m9.206983699s) [noise=true , address=104.36.82.36:4628 , piece_id=T4BALPITTFN333GZY324VOV4T375SH22QILJ5TKHIHCUA22W3ZOA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:05.601199152 (397h46m40.371840487s) [noise=true , address=104.36.82.19:4436 , piece_id=PECCQAP3QNU7NLMKZCYOPMPC3IJOYGAQKZ6K5MVV3GG6OAQWN57A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: <nil> Jul 9 22:57:17.760765568 (397h46m52.531406903s) [noise=true , address=45.91.95.2:4010 , piece_id=4ZAGVL5XGAWMN6D4TEB4K7X456SFA6LTGIEIC6DE5NXKE6ZGASZA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1XWYrjZ3FuoB117j5iRB2zxgeA3We8LHaUUHW6B8uiFkD88Ad7, address:45.91.95.2:4010): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760821976 (397h46m52.531463311s) [noise=true , address=45.91.95.2:4023 , piece_id=QUYHXAMIIOYA4KSXNTD7I6SMNAXBJ53Y2GIBF5T6PGU6KQWOC7JA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1GsGdvhc9tqAEYWDLDVG9Wrszda4EjYR9eTbFgepZNF2dEFazy, address:45.91.95.2:4023): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.760947976 (397h46m52.531589301s) [noise=true , address=45.91.95.4:4012 , piece_id=YVXX7S45KP5O3OG6HQBAOKEDER7WB7KQMQ7RXRG2GBWQGICO7YLQ, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:12kFYkJi2nr3EVpxZ6NVWcQwX8hRy3fXkfVmBNXjcmyPK2EoHMt, address:45.91.95.4:4012): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761017459 (397h46m52.531658794s) [noise=true , address=104.36.82.50:4027 , piece_id=PDDGX2BGMTE7KXOGNNMYE4576GF4YCVABK34LZNNFEOYYG7BVJ6A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1zEQhyuVxSHwdDUgDfcTog2W3oLBx4z9bYUMHpaaoLU1BTU9DQ, address:104.36.82.50:4027): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:57:17.761104556 (397h46m52.531745891s) [noise=true , address=45.91.95.3:4031 , piece_id=5B2NBC34NAQ5EIAC3AGDP2CWJC4ZRAGTUJ5SLP6CYCV2R4URFSRA, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:1bupgxDMcVUwTNB2uUHNeiG7pGira7jssXK1byhozFu2BA4ma1, address:45.91.95.3:4031): protocol: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44844->45.91.95.3:4031: use of closed network connection Jul 9 22:57:17.761152347 (397h46m52.531793672s) [noise=true , address=45.91.95.5:4030 , piece_id=LKHRM6UFDSW4NAACOH7M4EDWY2H2O3B6OA4MCV3MTA4MVMJKTR2A, seg_pos={0 0}, upload=318232]: Done uploading piece. err: ecclient: upload failed (node:126eg5nWqSdmnm9UUUpwEidVVFe9cgaNRraS2efAvtJcwg8qXfq, address:45.91.95.5:4030): protocol: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41376->45.91.95.5:4030: use of closed network connection Jul 9 22:57:17.761167226 (397h46m52.531808561s) [seg_pos={0 0}, upload=318232]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 9 22:57:17.992237959 (397h46m52.762879294s) [upload=318232]: Done uploading segments... Jul 9 22:57:17.992261995 (397h46m52.76290332s) [upload=318232]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 1m0.232616459s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X.........................X...............X...X.........] duration: 10.047638151s FAIL (2025-07-09 22:57:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.047662939s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X.........................X...............X...X........] duration: 6.03720384s FAIL (2025-07-09 22:56:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.037271929s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X.....................X...................................] duration: 25.539105733s FAIL (2025-07-09 22:56:34 UTC) orbiter warning: context deadline exceeded Duration: 25.539125031s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X.........................X...............X...X.......] duration: 11.059110704s FAIL (2025-07-09 22:56:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.059678551s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......X.........X...............................X..X.......] duration: 23.88272476s FAIL (2025-07-09 22:56:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8]: Done uploading segments... Jul 9 22:49:03.940425795 (397h38m38.71106713s) [upload=318138]: Upload finished. err: <nil> Jul 9 22:49:26.942685461 (397h39m1.713326806s) [upload=318145]: Starting upload Jul 9 22:49:26.942766596 (397h39m1.713407971s) [upload=318145]: Done starting upload Jul 9 22:49:26.942899851 (397h39m1.713541186s) [upload=318145]: Uploading segments... Jul 9 22:49:26.943078553 (397h39m1.713719888s) [upload=318145]: Got next segment. Inline: false Jul 9 22:49:26.943128258 (397h39m1.713769583s) [upload=318145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 22:49:26.943133378 (397h39m1.713774703s) [upload=318145]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:49:27.391221381 (397h39m2.161862716s) [seg_pos={0 0}, upload=318145]: Begin upload segment... Jul 9 22:49:27.391272879 (397h39m2.161914215s) [seg_pos={0 0}, upload=318145]: Done begin upload segment. Jul 9 22:49:27.391293639 (397h39m2.161934974s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.391339827 (397h39m2.161981152s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.391636765 (397h39m2.16227809s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.391776052 (397h39m2.162417387s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.391900591 (397h39m2.162541926s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392075054 (397h39m2.162716389s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392182890 (397h39m2.162824216s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392427849 (397h39m2.163069184s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392508023 (397h39m2.163149358s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392699018 (397h39m2.163340343s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392740437 (397h39m2.163381792s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392950770 (397h39m2.163592105s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393004523 (397h39m2.163645858s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393258058 (397h39m2.163899393s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393469623 (397h39m2.164110958s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393671368 (397h39m2.164312703s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393919603 (397h39m2.164560928s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.394128562 (397h39m2.164769897s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:28.541659683 (397h39m3.312301018s) [upload=318145]: Next returned nil segment Jul 9 22:49:28.541678779 (397h39m3.312320114s) [upload=318145]: Waiting for error group managing segments... Jul 9 22:49:29.222239328 (397h39m3.992880663s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.396477165 (397h39m4.1671185s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.396597805 (397h39m4.16723914s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.397196882 (397h39m4.167838227s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.398284352 (397h39m4.168925687s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.407089638 (397h39m4.177730973s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.414636957 (397h39m4.185278292s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:35.309098140 (397h39m10.079739475s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:41.926391228 (397h39m16.697032563s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:46.575760060 (397h39m21.346401385s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:50:02.779785025 (397h39m37.55042636s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:50:05.100996812 (397h39m39.871638147s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:50:20.559293926 (397h39m55.329935261s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:50:26.299711393 (397h40m1.070352728s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12Mwd9RRYqm3GpbmT4B2KnAd2bXKq7HopJCk3E3W4A79Hh2eypL, address:45.91.95.2:4031): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299775055 (397h40m1.07041639s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1rNDTtTWVFcGgWM4ZWJ4GKhoATGLJTeSUYnBDQKjeRasQ3U8cx, address:45.91.95.5:4031): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299782971 (397h40m1.070424356s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299809722 (397h40m1.070451057s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12rWJM3vC8XrUAfVaoqVZm1qzqcG8AqxLzncExPHEak5oo3QuPA, address:45.91.95.4:4030): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299880867 (397h40m1.070522283s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12USiZrzoPtP51hXzKeyNLg58wgmPCTAKMTvPb5du9CRKFu4FiJ, address:45.91.95.3:4007): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299921666 (397h40m1.070563001s) [seg_pos={0 0}, upload=318145]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 22:50:26.533201557 (397h40m1.303842892s) [upload=318145]: Done uploading segments... Jul 9 22:50:26.533239139 (397h40m1.303880474s) [upload=318145]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 23.88275072s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........X.........X...............................X..X......] duration: 12.0981512s FAIL (2025-07-09 22:55:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8]: Done uploading segments... Jul 9 22:49:03.940425795 (397h38m38.71106713s) [upload=318138]: Upload finished. err: <nil> Jul 9 22:49:26.942685461 (397h39m1.713326806s) [upload=318145]: Starting upload Jul 9 22:49:26.942766596 (397h39m1.713407971s) [upload=318145]: Done starting upload Jul 9 22:49:26.942899851 (397h39m1.713541186s) [upload=318145]: Uploading segments... Jul 9 22:49:26.943078553 (397h39m1.713719888s) [upload=318145]: Got next segment. Inline: false Jul 9 22:49:26.943128258 (397h39m1.713769583s) [upload=318145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 22:49:26.943133378 (397h39m1.713774703s) [upload=318145]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 22:49:27.391221381 (397h39m2.161862716s) [seg_pos={0 0}, upload=318145]: Begin upload segment... Jul 9 22:49:27.391272879 (397h39m2.161914215s) [seg_pos={0 0}, upload=318145]: Done begin upload segment. Jul 9 22:49:27.391293639 (397h39m2.161934974s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.391339827 (397h39m2.161981152s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.391636765 (397h39m2.16227809s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.391776052 (397h39m2.162417387s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.391900591 (397h39m2.162541926s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392075054 (397h39m2.162716389s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392182890 (397h39m2.162824216s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392427849 (397h39m2.163069184s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392508023 (397h39m2.163149358s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392699018 (397h39m2.163340343s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392740437 (397h39m2.163381792s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.392950770 (397h39m2.163592105s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393004523 (397h39m2.163645858s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393258058 (397h39m2.163899393s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393469623 (397h39m2.164110958s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393671368 (397h39m2.164312703s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.393919603 (397h39m2.164560928s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:27.394128562 (397h39m2.164769897s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Uploading piece... Jul 9 22:49:28.541659683 (397h39m3.312301018s) [upload=318145]: Next returned nil segment Jul 9 22:49:28.541678779 (397h39m3.312320114s) [upload=318145]: Waiting for error group managing segments... Jul 9 22:49:29.222239328 (397h39m3.992880663s) [noise=true , address=79.127.220.101:4076 , piece_id=BVSPRCWY544VM2MY22W42DF6TYRC3R7XWDZ56RSIEAXTECTMLFAA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.396477165 (397h39m4.1671185s) [noise=true , address=77.90.31.11:4575 , piece_id=K3K77Q5J6GJIRUP5MUMQXOHVJRQY5QAGPQ2F2MXS6RZXXFUZCDLA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.396597805 (397h39m4.16723914s) [noise=true , address=77.90.31.17:4798 , piece_id=M4RWB2ATFPRJYDVV56RAPKY74R3AAUYMURLU6URKMYSXY7MA6XSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.397196882 (397h39m4.167838227s) [noise=true , address=77.90.31.18:4822 , piece_id=AQMWVDBXJZSZ6YMQLWC4KC3MRIRJH3VJ44OOT6BX6BO2ALDUUHYA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.398284352 (397h39m4.168925687s) [noise=true , address=77.90.31.13:4642 , piece_id=5M544XRTSVY4RRFCPG2SDHMQQGMOYSZG24JNUCXTJ7XZ6DHMAQZA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.407089638 (397h39m4.177730973s) [noise=true , address=77.90.31.21:4898 , piece_id=B6F2GYBK62AOTSOKBF2HXZEWWRKOIHSTJ344URSKMB372RWUGESA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:29.414636957 (397h39m4.185278292s) [noise=true , address=77.90.31.19:4856 , piece_id=4RJKOYYOTU3ULBFD63HAGMROVNSHW4N7VKBPEFXFZKH4QTEFEETA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:35.309098140 (397h39m10.079739475s) [noise=true , address=79.127.219.38:4022 , piece_id=U3SCEOK3VJ5WZBPIWQCNB53J2QRVF5JMMCVMOUQXBSHS5G5FYNPQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:41.926391228 (397h39m16.697032563s) [noise=true , address=104.36.82.41:4772 , piece_id=VIZA6ZGIR7BQY5X7K242GRF6QID43D6DGZA4NK2EMNYJOPNL5BSQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:49:46.575760060 (397h39m21.346401385s) [noise=true , address=104.36.82.39:4743 , piece_id=AYESUTGL42X22LJHWJZ7Z7YVAH5FU733XFHZLPUUZFOQHJDKMNSA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:50:02.779785025 (397h39m37.55042636s) [noise=true , address=104.36.82.22:4527 , piece_id=BIPYWHWT2WBBBVYZBNURDMCDD67FIPFBUXH6HLU64WGY36ZYSFRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:50:05.100996812 (397h39m39.871638147s) [noise=true , address=104.36.82.44:4884 , piece_id=WY3MYTDBCQXDIQUUEKTKACNAET6UV3M74W6AFOI5PTRGW444GAVQ, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:50:20.559293926 (397h39m55.329935261s) [noise=true , address=104.36.82.17:4343 , piece_id=FK7JL2DODS5LLZVWBGG73CFXGOWNGUVPNZB43VFN2NB6EDRJJGFA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: <nil> Jul 9 22:50:26.299711393 (397h40m1.070352728s) [noise=true , address=45.91.95.2:4031 , piece_id=GKVSJKE3WTRYUJFHSRFGMUBBAJVLAOP7ML5HSGAF3EMYEBMURI7Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12Mwd9RRYqm3GpbmT4B2KnAd2bXKq7HopJCk3E3W4A79Hh2eypL, address:45.91.95.2:4031): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299775055 (397h40m1.07041639s) [noise=true , address=45.91.95.5:4031 , piece_id=LPZDCJHVPP57LTPBYUME3EQ2ZVHOQSRCJ3DNXJRSXX2VUKJUHS4Q, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1rNDTtTWVFcGgWM4ZWJ4GKhoATGLJTeSUYnBDQKjeRasQ3U8cx, address:45.91.95.5:4031): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299782971 (397h40m1.070424356s) [noise=true , address=64.49.13.169:4706 , piece_id=QH53VD6AFLOPSDMU5SLTO2IU3VWBXTFAYVBIEW2ANC77OEFGDFTA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299809722 (397h40m1.070451057s) [noise=true , address=45.91.95.4:4030 , piece_id=FHCM52JJ5C6IGBP5LZUKYZLBFGJDHZQJ2F3IAY4IOQKQAFNY46HA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12rWJM3vC8XrUAfVaoqVZm1qzqcG8AqxLzncExPHEak5oo3QuPA, address:45.91.95.4:4030): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299880867 (397h40m1.070522283s) [noise=true , address=45.91.95.3:4007 , piece_id=2SAQJ6S6XI3KVLYXCEN2IFO2KGKF2GTYANXFQ7KDDJ3EXP2W3CRA, seg_pos={0 0}, upload=318145]: Done uploading piece. err: ecclient: upload failed (node:12USiZrzoPtP51hXzKeyNLg58wgmPCTAKMTvPb5du9CRKFu4FiJ, address:45.91.95.3:4007): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 22:50:26.299921666 (397h40m1.070563001s) [seg_pos={0 0}, upload=318145]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 22:50:26.533201557 (397h40m1.303842892s) [upload=318145]: Done uploading segments... Jul 9 22:50:26.533239139 (397h40m1.303880474s) [upload=318145]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 12.098175956s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)