Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.40327289s FAIL (2025-10-16 18:09:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.403305162s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.182569821s FAIL (2025-10-16 18:09:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.18260118s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.097506694s FAIL (2025-10-16 18:09:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.097569845s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.142092073s FAIL (2025-10-16 18:09:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.142131028s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.186819654s FAIL (2025-10-16 18:09:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.186837908s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.......X..............X.....................................] duration: 10.221524317s FAIL (2025-10-16 18:09:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 10.221557509s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.302444486s FAIL (2025-10-16 18:09:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.302478803s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.331343582s FAIL (2025-10-16 18:08:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.331371854s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.064969077s FAIL (2025-10-16 18:08:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.065010866s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.383208851s FAIL (2025-10-16 18:08:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.383261772s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [........X..............X....................................] duration: 6.691663695s FAIL (2025-10-16 18:08:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.69169721s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.........X..............X...................................] duration: 7.094306803s FAIL (2025-10-16 18:08:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 7.094341098s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.366203981s FAIL (2025-10-16 18:08:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.366235883s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.145039143s FAIL (2025-10-16 18:08:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.145071897s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.051400368s FAIL (2025-10-16 18:07:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.051430336s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..........X..............X..................................] duration: 6.749274979s FAIL (2025-10-16 18:07:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.749310426s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.337902211s FAIL (2025-10-16 18:07:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.337932959s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.22709753s FAIL (2025-10-16 18:07:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.227127185s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...........X..............X.................................] duration: 6.79938135s FAIL (2025-10-16 18:07:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.799415064s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.128744346s FAIL (2025-10-16 18:07:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.128848694s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.155439819s FAIL (2025-10-16 18:07:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.155473021s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.586693948s FAIL (2025-10-16 18:07:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.586793948s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.065939327s FAIL (2025-10-16 18:06:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.065987248s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.136708503s FAIL (2025-10-16 18:06:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.136732558s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [............X..............X................................] duration: 7.717804227s FAIL (2025-10-16 18:06:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 7.717839374s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [.............X..............X...............................] duration: 8.245094049s FAIL (2025-10-16 18:06:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 8.245133284s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.339344346s FAIL (2025-10-16 18:06:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.339380044s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.519258777s FAIL (2025-10-16 18:06:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.519291419s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.283886154s FAIL (2025-10-16 18:06:06 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.283913646s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.244951745s FAIL (2025-10-16 18:06:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.244988556s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [..............X..............X..............................] duration: 7.07618279s FAIL (2025-10-16 18:05:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 7.076227004s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.078026912s FAIL (2025-10-16 18:05:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.07807356s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.205086725s FAIL (2025-10-16 18:05:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.205121171s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.047008558s FAIL (2025-10-16 18:05:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.047046872s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.526041463s FAIL (2025-10-16 18:05:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.526073745s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [...............X..............X.............................] duration: 6.152751276s FAIL (2025-10-16 18:04:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 6.152777846s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.172768574s FAIL (2025-10-16 18:04:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.172801918s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.328744471s FAIL (2025-10-16 18:04:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.328776722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.217988368s FAIL (2025-10-16 18:04:31 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.218022071s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.239916763s FAIL (2025-10-16 18:04:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.239999501s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.347955713s FAIL (2025-10-16 18:04:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.347992241s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.308988839s FAIL (2025-10-16 18:04:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.30902664s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.351122511s FAIL (2025-10-16 18:03:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.351154612s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 2 out of 60 is failed (threshold=3) [................X..............X............................] duration: 5.608111254s FAIL (2025-10-16 18:03:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 23) ======LOGS====== e=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939015459 (556h9m4.401107952s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.939569114 (556h9m4.401661607s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940098503 (556h9m4.402190996s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.940843681 (556h9m4.402936174s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.941577598 (556h9m4.403670082s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.942334921 (556h9m4.404427404s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:18.945888277 (556h9m4.40798078s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Uploading piece... Oct 16 17:40:19.833989424 (556h9m5.296081907s) [upload=489206]: Next returned nil segment Oct 16 17:40:19.834010063 (556h9m5.296102546s) [upload=489206]: Waiting for error group managing segments... Oct 16 17:40:20.233442816 (556h9m5.695535299s) [noise=true , address=89.149.201.147:4016 , piece_id=HUTENU3DY5KXWA6P3TJ4GQY74K4EZD3OSV4ITAYIPFHHKBK4GAZQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.262739020 (556h9m5.724831503s) [noise=true , address=89.149.201.147:4004 , piece_id=I4LIWTSLHF3THHJHUN3SHENKDZETC6OJES2THU6EHZKDRWXKFINQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.265174598 (556h9m5.727267081s) [noise=true , address=89.149.201.147:4005 , piece_id=LRI372M6KW5BEFCFYTU44HI7QC7L72UI5SACYZRSO4IX4JFQ6TLA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.286810777 (556h9m5.748903261s) [noise=true , address=89.149.201.147:4023 , piece_id=XJ2ZLUEQPKZUNZPNOX7RMX3FZRPYRYBCT37FZXSFN7HXZYAWVQNQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.310521828 (556h9m5.772614311s) [noise=true , address=89.149.201.147:4009 , piece_id=H6RP72BYM24CNAZSGW4ZBK5A5W7UD2H4XE54XEZOQYQ26VYPKSHQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.319192228 (556h9m5.781284701s) [noise=true , address=89.149.201.147:4017 , piece_id=Z5IFTBBVQTC45Z6OFY4XTPN5Y5TAEL33BACVPY7XRQ6GWQVLDEJQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.752645442 (556h9m6.214737925s) [noise=true , address=185.167.204.206:4044 , piece_id=PUSCNT7LC4C36SZ5KJFVELMB4BFAGIR4ZPDWZASIWK2GQ3MERHCQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.779360633 (556h9m6.241453106s) [noise=true , address=185.167.204.206:4072 , piece_id=I2DM2NJXBNFNF4QDMZYOETIILC6OA566D3YX3HJHC65J2TLSTP4A, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.806951290 (556h9m6.269043773s) [noise=true , address=185.167.204.206:4057 , piece_id=WMJ76VTT6EVWMT76IQA4ZFMAJFILZG7DNYDPLECF5WYGSEXEXR3Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.858902605 (556h9m6.320995088s) [noise=true , address=185.167.204.206:4054 , piece_id=SNY7QWPZQY5XWWQB73GFMWSB2L6T4LQM4HKUMHM4AZIHGXSFDFYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.874868968 (556h9m6.336961451s) [noise=true , address=185.167.204.206:4064 , piece_id=XX5TASB4R7TMQ6AIBO4JYCZKFHN34DKYIMDJ2O3SHS5Z3PVJY4MA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.875491774 (556h9m6.337584257s) [noise=true , address=185.167.204.205:4017 , piece_id=N57HTKOPE6HH5I6NUVCIG2IRH2F7TUPGCWEI6L4R2LOJPQWW4X6Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:20.890675426 (556h9m6.352767909s) [noise=true , address=185.167.204.205:4020 , piece_id=KFLWAD5QY3BTZT7MNVQ4YXFGWWFLZLJXOLCJD5KD2EGI6X32ARYA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.517656051 (556h9m6.979748534s) [noise=true , address=185.167.204.205:4036 , piece_id=FGEVAOYUTMOT5OZMSEFGSUOQNAPG3GFMQVZ46X6TIIZD2CQV6WAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.538778642 (556h9m7.000871125s) [noise=true , address=185.167.204.205:4027 , piece_id=L2UKHWTCP7SKH243BWTI6ICX5VB45D6626BPXKGX2RFWT5243MFQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539014291 (556h9m7.001106764s) [noise=true , address=185.167.204.206:4062 , piece_id=S5IE5L2W6OD2LZF72NXKR4YSOJTTHG72SSMVQLMXZAJ2KXCDK2KA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.539980991 (556h9m7.002073474s) [noise=true , address=185.167.204.206:4048 , piece_id=G3TH2LW7MQC5EN33AXEJKQ7R7AROYNSFR4RH7M7Q7I3P3L3MB5VA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543539668 (556h9m7.005632152s) [noise=true , address=185.167.204.206:4066 , piece_id=GMFOLROCRIUMHEG57G65CTIEPCYMPRW57QZGLO2AWYI3ODUONXQA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.543649087 (556h9m7.00574157s) [noise=true , address=185.167.204.206:4051 , piece_id=LNKN57W67HA24PWVUKUWFAE3HRR3SDVUUI6YNUKBJSR7CQBR3RFA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.552180291 (556h9m7.014272775s) [noise=true , address=185.167.204.205:4004 , piece_id=JBEJHK4GXQHXBR4LVLJEDTZ7F76VMQNIS3TKIO7RDPA4OUTAXASQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.562425531 (556h9m7.024518014s) [noise=true , address=185.167.204.205:4015 , piece_id=JMHSTDV3P4HVQT3WB7TZCOY4ICAR5EHJEOSJM5EFTV3VNVIKATAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:21.581114879 (556h9m7.043207362s) [noise=true , address=185.167.204.205:4008 , piece_id=BZATZ4AGUEEESZFOVUNGCYXE37BKLBMLV66VT3EHY2MVFA5VA63Q, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:40:23.178696305 (556h9m8.640788788s) [noise=true , address=185.167.204.205:4022 , piece_id=NRKA4JZF7EX3MEEVVERFYZWQA6AQY3C5EASAMTAAJYKJ6QTXNAXQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: <nil> Oct 16 17:41:18.183575160 (556h10m3.645667643s) [noise=true , address=81.171.14.70:4043 , piece_id=3ORLBCAZ7XQ2ANVRZ3QPTBXNBTWCP4KNUZQCZCSEA5JSQXOQ63FA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52680->81.171.14.70:4043: use of closed network connection Oct 16 17:41:18.183784959 (556h10m3.645877442s) [noise=true , address=81.171.14.70:4031 , piece_id=Q33ZXVTW4LISVDK7V7WVGSL7WXDACEKS3BDVKVZJ37WPURSM6NMQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: rpc: dial tcp 81.171.14.70:4031: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.183845906 (556h10m3.645938389s) [noise=true , address=81.171.14.70:4026 , piece_id=W5QZGKIC5YCIRPKVZYL7V44ZM5FXEBKMMWADXPYFE2BZ5VCUVYQQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56926->81.171.14.70:4026: use of closed network connection Oct 16 17:41:18.183905408 (556h10m3.645997891s) [noise=true , address=81.171.14.70:4039 , piece_id=BO34I6O7K6Y4ZPUZ24MQM64GRRUW3I5KY466GXUW75FLB62QLTAQ, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40332->81.171.14.70:4039: use of closed network connection Oct 16 17:41:18.184184199 (556h10m3.646276682s) [noise=true , address=81.171.14.70:4042 , piece_id=XHJCJJHE4MYEKMRGXQXKXEIWXFVRZTNQTLRVWOZTA53PFK5YVXAA, seg_pos={0 0}, upload=489206]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40438->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 16 17:41:18.184212784 (556h10m3.646305257s) [seg_pos={0 0}, upload=489206]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil> Oct 16 17:41:18.381927960 (556h10m3.844020453s) [upload=489206]: Done uploading segments... Oct 16 17:41:18.381973867 (556h10m3.84406634s) [upload=489206]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23) Duration: 5.608142984s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.364662672s FAIL (2025-10-16 18:03:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.364694692s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.114885472s FAIL (2025-10-16 18:03:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 16 17:30:42.766933210 (555h59m28.229025693s) [upload=489069]: Starting upload Oct 16 17:30:42.767033852 (555h59m28.229126325s) [upload=489069]: Done starting upload Oct 16 17:30:42.767107392 (555h59m28.229199875s) [upload=489069]: Uploading segments... Oct 16 17:30:42.767155244 (555h59m28.229247727s) [upload=489069]: Got next segment. Inline: true Oct 16 17:30:42.767161455 (555h59m28.229253938s) [upload=489069]: Waiting for error group managing segments... Oct 16 17:30:42.767213445 (555h59m28.229305928s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:42.767220328 (555h59m28.229312811s) [upload=489069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Oct 16 17:30:42.767224846 (555h59m28.229317329s) [upload=489069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 16 17:31:12.052776739 (555h59m57.514869222s) [upload=489069]: Done uploading segments... Oct 16 17:31:12.052807918 (555h59m57.514900401s) [upload=489069]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.114920138s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.774107591s FAIL (2025-10-16 18:03:20 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.774144249s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.141545298s FAIL (2025-10-16 18:03:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.141573712s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.29392146s FAIL (2025-10-16 18:03:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.293982436s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.235559266s FAIL (2025-10-16 18:02:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Oct 16 17:30:41.047160219 (555h59m26.509252702s) [upload=489068]: Starting upload Oct 16 17:30:41.047238237 (555h59m26.50933072s) [upload=489068]: Done starting upload Oct 16 17:30:41.047341244 (555h59m26.509433717s) [upload=489068]: Uploading segments... Oct 16 17:30:41.047497971 (555h59m26.509590455s) [upload=489068]: Got next segment. Inline: false Oct 16 17:30:41.047548318 (555h59m26.509640801s) [upload=489068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 16 17:30:41.047573415 (555h59m26.509665898s) [upload=489068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 16 17:31:10.319457462 (555h59m55.781549976s) [upload=489068]: Next segment err: context deadline exceeded Oct 16 17:31:10.319703441 (555h59m55.781795934s) [upload=489068]: Done uploading segments... Oct 16 17:31:10.319732436 (555h59m55.781824909s) [upload=489068]: Upload finished. err: context deadline exceeded Duration: 6.235593883s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)