Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.325857786s FAIL (2025-07-01 13:49:36 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 6.325885028s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.647223662s FAIL (2025-07-01 13:48:56 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.647247247s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.526488413s FAIL (2025-07-01 13:48:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.526508804s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.595423326s FAIL (2025-07-01 13:48:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.59545136s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.547989153s FAIL (2025-07-01 13:47:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.548018408s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.671929193s FAIL (2025-07-01 13:47:13 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.671957788s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.624336151s FAIL (2025-07-01 13:47:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.624381828s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.636347429s FAIL (2025-07-01 13:46:27 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.636378069s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.543444385s FAIL (2025-07-01 13:46:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.54346746s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.654014518s FAIL (2025-07-01 13:46:01 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.654039976s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.489804441s FAIL (2025-07-01 13:45:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.489825662s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.785897337s FAIL (2025-07-01 13:45:08 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.785929869s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.531640512s FAIL (2025-07-01 13:44:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.53166059s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.799050558s FAIL (2025-07-01 13:43:58 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.799082038s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.552825584s FAIL (2025-07-01 13:43:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.552851834s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.777667379s FAIL (2025-07-01 13:42:54 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.777692958s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.123764905s FAIL (2025-07-01 13:42:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.123859105s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.765809475s FAIL (2025-07-01 13:42:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.765883628s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.588061153s FAIL (2025-07-01 13:42:16 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.588085749s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.189497027s FAIL (2025-07-01 13:42:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.189528619s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.635310849s FAIL (2025-07-01 13:42:00 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.635343502s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.543590424s FAIL (2025-07-01 13:41:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.543632323s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.873896648s FAIL (2025-07-01 13:41:28 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.873921796s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.227493561s FAIL (2025-07-01 13:40:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.227531654s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.009657457s FAIL (2025-07-01 13:40:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.009680242s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.634809825s FAIL (2025-07-01 13:40:16 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.634835625s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.934654054s FAIL (2025-07-01 13:39:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.934670535s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.818724509s FAIL (2025-07-01 13:39:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.818739097s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.733946107s FAIL (2025-07-01 13:39:18 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.733976404s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.068826928s FAIL (2025-07-01 13:38:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.068865223s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.674099935s FAIL (2025-07-01 13:38:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.674164838s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.524095634s FAIL (2025-07-01 13:38:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.524133095s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.701736595s FAIL (2025-07-01 13:38:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.701759338s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.789181803s FAIL (2025-07-01 13:37:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.789200529s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.493629016s FAIL (2025-07-01 13:37:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.493649737s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.709706263s FAIL (2025-07-01 13:37:27 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.709732412s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.683605037s FAIL (2025-07-01 13:37:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.683656655s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.293273281s FAIL (2025-07-01 13:37:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.29328856s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.704030204s FAIL (2025-07-01 13:36:47 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.704070603s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.399692828s FAIL (2025-07-01 13:36:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.399707396s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.248036327s FAIL (2025-07-01 13:36:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.248057808s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.977147918s FAIL (2025-07-01 13:35:38 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.977180731s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.136382854s FAIL (2025-07-01 13:35:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.136414372s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.676908147s FAIL (2025-07-01 13:35:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.677064685s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 9.480637175s FAIL (2025-07-01 13:35:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.480660388s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.050893995s FAIL (2025-07-01 13:34:57 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 6.050919462s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.762829615s FAIL (2025-07-01 13:34:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.762849774s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.707731613s FAIL (2025-07-01 13:34:27 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== ading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Jul 1 13:23:32.267561192 (196h13m7.038202527s) [seg_pos={0 0}, upload=156954]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:23:32.267626126 (196h13m7.038267511s) [upload=156954]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:23:32.267633580 (196h13m7.038274955s) [upload=156954]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:23:32.564479871 (196h13m7.335121206s) [upload=156954]: Done uploading segments... Jul 1 13:23:32.564498377 (196h13m7.335139702s) [upload=156954]: Upload finished. err: <nil> Jul 1 13:24:29.560888309 (196h14m4.331529644s) [upload=156965]: Starting upload Jul 1 13:24:29.560940799 (196h14m4.331582124s) [upload=156965]: Done starting upload Jul 1 13:24:29.561122977 (196h14m4.331764312s) [upload=156965]: Uploading segments... Jul 1 13:24:29.561173023 (196h14m4.331814358s) [upload=156965]: Got next segment. Inline: false Jul 1 13:24:29.561201949 (196h14m4.331843284s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 1 13:24:29.561206607 (196h14m4.331847942s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 1 13:24:29.805112380 (196h14m4.575753715s) [seg_pos={0 0}, upload=156965]: Begin upload segment... Jul 1 13:24:29.805161915 (196h14m4.57580324s) [seg_pos={0 0}, upload=156965]: Done begin upload segment. Jul 1 13:24:29.805196952 (196h14m4.575838287s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805367638 (196h14m4.576008973s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.805862585 (196h14m4.57650392s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806151198 (196h14m4.576792533s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806405574 (196h14m4.577046909s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806538769 (196h14m4.577180114s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806644702 (196h14m4.577286037s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.806872938 (196h14m4.577514273s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807087589 (196h14m4.577728924s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807106134 (196h14m4.577747469s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807323490 (196h14m4.577964825s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.807552468 (196h14m4.578193803s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808051863 (196h14m4.578693198s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.808466777 (196h14m4.579108122s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.809998528 (196h14m4.580639853s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:29.810327147 (196h14m4.580968482s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Uploading piece... Jul 1 13:24:31.567215995 (196h14m6.33785733s) [upload=156965]: Next returned nil segment Jul 1 13:24:31.567238518 (196h14m6.337879853s) [upload=156965]: Waiting for error group managing segments... Jul 1 13:24:31.849330083 (196h14m6.619971418s) [noise=true , address=45.91.116.5:5297 , piece_id=HL3OFARGL3745TS3HQF6A52ILFP2NKN4U57TQIJXGSCFWQ3ZM5IQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.868737580 (196h14m6.639378915s) [noise=true , address=45.91.116.4:4940 , piece_id=H5GRLBRIHR5O5PURBAQ4YOBHBZK7SLUVOQWJD6ZY3MCHUATJEDQQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.870984328 (196h14m6.641625663s) [noise=true , address=45.91.116.6:4975 , piece_id=4JEPXGYVKTWUGZBT6JZWJFYVO4GSKEKWYJEBIFW3K7ORCPFARGQA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.890536413 (196h14m6.661177738s) [noise=true , address=45.91.116.14:4189 , piece_id=4B6UKVAS4NSIESCQ3VSVJWZSZVZSSPD6CP3TU4QHYPGZSWZ4WN7A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.904306657 (196h14m6.674947992s) [noise=true , address=77.90.31.10:4518 , piece_id=DRWBG6EFBMV5LBR4WE3RTFSABHVVQW47KWQH7I3J5EUBZLI3RMSQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.941272175 (196h14m6.7119135s) [noise=true , address=45.91.116.4:4938 , piece_id=KVH7H24CFUSL3763MANE4ASNVTNR7PBQSNNKFNRNB754XYH5GQXA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:31.943798498 (196h14m6.714439833s) [noise=true , address=77.90.31.17:4770 , piece_id=ODZ5ZLOHG5LMNKZLZKKEUNSADF6OREA564L3WMGUTXAIG4CK46QA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.011600759 (196h14m6.782242094s) [noise=true , address=77.90.31.15:4699 , piece_id=O7BT75G4Y2Y453MSKTDXQXGHTJW7MK4JHNRNAHHGBXWQIQ4BMMVQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.130468763 (196h14m6.901110098s) [noise=true , address=104.36.82.15:4259 , piece_id=KRLCC26TFIA2HHHICUFWW4JEUWMUUDZCS7JIBMAATXZTTXDUNLTA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.284626620 (196h14m7.055267955s) [noise=true , address=104.36.82.14:4222 , piece_id=BRRPFTGE4VQE7FAVBCCFSSOC5SVJ3MAZAYMX4FW2SWEGJNXY4O6A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.608427086 (196h14m7.379068421s) [noise=true , address=104.36.82.8:4970 , piece_id=23YGAZWYSDY4UM2MVMBXOEY4PRQPTND3ZOAYD4XXOC5IDIJPNVDA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.619299124 (196h14m7.389940449s) [noise=true , address=104.36.82.7:4911 , piece_id=7H7L7EW63TFAOURC55GSD7ZXHCH23VU575QU4FFROILFBTC52ANA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624723432 (196h14m7.395364767s) [noise=true , address=104.36.82.13:4187 , piece_id=KX55KOFMDJ2GJ5R3T75VZCELDRCJBZ6L4TZFPOVEX2JPPNOAB66A, seg_pos={0 0}, upload=156965]: Done uploading piece. err: <nil> Jul 1 13:24:32.624745435 (196h14m7.39538677s) [seg_pos={0 0}, upload=156965]: Segment reached optimal threshold of 13 pieces. Jul 1 13:24:32.624861536 (196h14m7.395502872s) [noise=true , address=104.36.82.10:4078 , piece_id=IKGLBYC7NJXPNXHVOAJCZKUAOU4DVLZMG3E3W2FOCTYTI2QW2MLQ, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624920279 (196h14m7.395561614s) [noise=true , address=104.36.82.6:4875 , piece_id=634PATJDQZFE5SPLJKHMJ6RFF6HEXMCM6EZF6CR5AXQJVV3UZVNA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF Jul 1 13:24:32.624980784 (196h14m7.395622119s) [noise=true , address=104.36.82.2:4731 , piece_id=SUVIT3Q2373VGKIRIRHSE5A5HW4MICHKXDY6DTF47KQNQ4YHMGRA, seg_pos={0 0}, upload=156965]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44914->104.36.82.2:4731: use of closed network connection Jul 1 13:24:32.624996214 (196h14m7.395637549s) [seg_pos={0 0}, upload=156965]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 1 13:24:32.625055297 (196h14m7.395696622s) [upload=156965]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 1 13:24:32.625059255 (196h14m7.39570059s) [upload=156965]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 1 13:24:32.928530190 (196h14m7.699171545s) [upload=156965]: Done uploading segments... Jul 1 13:24:32.928576809 (196h14m7.699218184s) [upload=156965]: Upload finished. err: <nil> Duration: 5.707759738s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.658434104s FAIL (2025-07-01 13:34:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.658452909s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.501576016s FAIL (2025-07-01 13:34:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.501602858s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)