Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 7.152066133s FAIL (2025-09-13 14:32:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 7.152096311s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X......X...........X......X.XX.XX.XX....................X.] duration: 6.502115299s FAIL (2025-09-13 14:31:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.502139998s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.932870638s FAIL (2025-09-13 14:31:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 5.932906976s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X......X...........X......X.XX.XX.XX....................X] duration: 30.00136861s FAIL (2025-09-13 14:30:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001393457s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.340614367s FAIL (2025-09-13 14:30:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.340633202s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X......X...........X......X.XX.XX.XX....................] duration: 6.641611703s FAIL (2025-09-13 14:30:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.641638747s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.678840299s FAIL (2025-09-13 14:29:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.678904352s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 7.392739766s FAIL (2025-09-13 14:29:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 7.392764193s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X......X...........X......X.XX.XX.XX...................] duration: 6.896912688s FAIL (2025-09-13 14:29:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.89693504s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.368264618s FAIL (2025-09-13 14:28:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.368291669s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X......X...........X......X.XX.XX.XX..................] duration: 6.824919299s FAIL (2025-09-13 14:28:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.82494033s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.28888233s FAIL (2025-09-13 14:28:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.288909142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 8.113528591s FAIL (2025-09-13 14:27:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 8.113557234s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X......X...........X......X.XX.XX.XX.................] duration: 6.707327015s FAIL (2025-09-13 14:27:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.707348174s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.115894344s FAIL (2025-09-13 14:27:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.115930442s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........X......X...........X......X.XX.XX.XX................] duration: 6.63778432s FAIL (2025-09-13 14:26:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.637803137s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.112593193s FAIL (2025-09-13 14:26:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.112619423s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 8.101240773s FAIL (2025-09-13 14:26:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 8.101273103s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.........X......X...........X......X.XX.XX.XX...............] duration: 6.81776508s FAIL (2025-09-13 14:25:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.817788575s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.275443125s FAIL (2025-09-13 14:25:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.275468955s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..........X......X...........X......X.XX.XX.XX..............] duration: 6.679352801s FAIL (2025-09-13 14:24:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.679371528s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.658215483s FAIL (2025-09-13 14:24:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.658245561s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...........X......X...........X......X.XX.XX.XX.............] duration: 6.62721456s FAIL (2025-09-13 14:23:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.627231582s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 8.466784859s FAIL (2025-09-13 14:23:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 8.466861796s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [............X......X...........X......X.XX.XX.XX............] duration: 6.797793671s FAIL (2025-09-13 14:23:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.797858575s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.987543952s FAIL (2025-09-13 14:23:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.987562446s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.............X......X...........X......X.XX.XX.XX...........] duration: 6.75996276s FAIL (2025-09-13 14:22:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.759989882s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.953178949s FAIL (2025-09-13 14:21:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.953206272s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..............X......X...........X......X.XX.XX.XX..........] duration: 6.869159376s FAIL (2025-09-13 14:21:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.869175297s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 9.527559821s FAIL (2025-09-13 14:20:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 9.52760156s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...............X......X...........X......X.XX.XX.XX.........] duration: 6.632393265s FAIL (2025-09-13 14:20:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.632407954s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.03884489s FAIL (2025-09-13 14:20:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.038879486s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [................X......X...........X......X.XX.XX.XX........] duration: 6.914379969s FAIL (2025-09-13 14:20:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.914426017s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 7.070690034s FAIL (2025-09-13 14:19:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 7.070713679s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.................X......X...........X......X.XX.XX.XX.......] duration: 6.834278135s FAIL (2025-09-13 14:19:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.834302321s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................X......X...........X......X.XX.XX.XX......] duration: 6.822898799s FAIL (2025-09-13 14:18:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.822914879s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.235453463s FAIL (2025-09-13 14:18:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.23547711s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...................X......X...........X......X.XX.XX.XX.....] duration: 6.697587032s FAIL (2025-09-13 14:17:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.697611298s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 9.940486519s FAIL (2025-09-13 14:17:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 9.940522399s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 9.111599813s FAIL (2025-09-13 14:16:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 9.111624981s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....................X......X...........X......X.XX.XX.XX....] duration: 6.674612701s FAIL (2025-09-13 14:16:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.674630044s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....................X......X...........X......X.XX.XX.XX...] duration: 6.663854956s FAIL (2025-09-13 14:16:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.663925499s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.915679047s FAIL (2025-09-13 14:16:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 5.91571173s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.537919457s FAIL (2025-09-13 14:15:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.537946979s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......................X......X...........X......X.XX.XX.XX..] duration: 7.158449141s FAIL (2025-09-13 14:15:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.158471584s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 7.398726774s FAIL (2025-09-13 14:14:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 7.398754657s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......................X......X...........X......X.XX.XX.XX.] duration: 6.648203613s FAIL (2025-09-13 14:14:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.648224134s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........................X......X...........X......X.XX.XX.XX] duration: 30.000883973s FAIL (2025-09-13 14:14:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000896176s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.43622165s FAIL (2025-09-13 14:14:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 13 14:02:14.203891398 (168h33m50.243464518s) [upload=148215]: Starting upload Sep 13 14:02:14.203965258 (168h33m50.243538378s) [upload=148215]: Done starting upload Sep 13 14:02:14.204183566 (168h33m50.243756696s) [upload=148215]: Uploading segments... Sep 13 14:02:14.204236577 (168h33m50.243809708s) [upload=148215]: Got next segment. Inline: false Sep 13 14:02:14.204272487 (168h33m50.243845607s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 13 14:02:14.204279320 (168h33m50.24385244s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 13 14:02:14.478745074 (168h33m50.518318194s) [seg_pos={0 0}, upload=148215]: Begin upload segment... Sep 13 14:02:14.478791794 (168h33m50.518364914s) [seg_pos={0 0}, upload=148215]: Done begin upload segment. Sep 13 14:02:14.478816190 (168h33m50.51838931s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.478884270 (168h33m50.51845739s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479165047 (168h33m50.518738167s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479430475 (168h33m50.519003605s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479738444 (168h33m50.519311565s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.479981009 (168h33m50.519554129s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480306732 (168h33m50.519879862s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480549777 (168h33m50.520122897s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480762855 (168h33m50.520335975s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.480986252 (168h33m50.520559373s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481195963 (168h33m50.520769083s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.481406867 (168h33m50.520979997s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482280308 (168h33m50.521853438s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.482750468 (168h33m50.522323588s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483057004 (168h33m50.522630124s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:14.483400131 (168h33m50.522973261s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Uploading piece... Sep 13 14:02:16.289339576 (168h33m52.328912696s) [upload=148215]: Next returned nil segment Sep 13 14:02:16.289357070 (168h33m52.32893019s) [upload=148215]: Waiting for error group managing segments... Sep 13 14:02:16.579195427 (168h33m52.618768547s) [noise=true , address=45.91.116.4:4939 , piece_id=JDRTRM73DIN5XFPDZGN4ZSNQCVVBWZHLEAYCLTFWQ2C37MW2W4EQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.605559675 (168h33m52.645132805s) [noise=true , address=77.90.31.10:4517 , piece_id=3NTWKNXYHRF3RDBHEKWPPYELHMMCABFC62AT5JCFIZN6OKVQ7GOQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.608696767 (168h33m52.648269887s) [noise=true , address=45.91.116.6:4975 , piece_id=DYYUABS6FDLIBZXKWDY5SIPNQOLO4OGOWPBW4G6AKWVHPQYMRPYQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.618230921 (168h33m52.657804041s) [noise=true , address=45.91.116.6:4973 , piece_id=QAOJ2Z3P6YWTEAAKU3KVQ2NSKSHIY2SNC6F3CEZIISORJK6L23RA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.627155348 (168h33m52.666728468s) [noise=true , address=45.91.116.9:4261 , piece_id=VMM64YIESCZUQKDFMXSK3KWYCDH4G34ODWOU6USFNC47U7SIVDXQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.634210989 (168h33m52.673784109s) [noise=true , address=77.90.31.12:4592 , piece_id=IL5I62NLJZ5RLSMAACMH4YF3BB65RE2MFNLSFX6VMU5675HLSZ3Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.640163388 (168h33m52.679736508s) [noise=true , address=77.90.31.17:4770 , piece_id=5QDYJLYH5TB4636UF3TROXMFDI6W2RKZ57JQTGJ6MUIU7PDFLSFQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.648466966 (168h33m52.688040086s) [noise=true , address=45.91.116.5:4298 , piece_id=JZXM4KHQKURJ27LWX2SXT2FJFXOZYBB2D4X2U4XDNN67LC6XDI3A, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.650630778 (168h33m52.690203898s) [noise=true , address=45.91.116.8:4047 , piece_id=E4DDB6TECPVZVBMN6S2MJU4EAMBUYUJBMHUKLHAEC2IGAABICUFA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661168200 (168h33m52.70074133s) [noise=true , address=77.90.31.21:4877 , piece_id=KMXCKWIENBN4MMS2YROUQLG2NSXJAYRRZ6XEEBRPATY4BWRFJ2AA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.661946700 (168h33m52.70151983s) [noise=true , address=45.91.116.13:4155 , piece_id=NI2QHB5NIEOK3KVZHJOLEWTU2FICU3ZQ56XHTAVHDX45ZGZ6UOHQ, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.673423199 (168h33m52.712996319s) [noise=true , address=45.91.116.8:4048 , piece_id=BV4BO5HTLVTQTJDGCEUUAPXWHOWU5MI67B3FJQA64JRUDDJVUZLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744308406 (168h33m52.783881526s) [noise=true , address=45.91.116.14:4191 , piece_id=3ETQLLEBHMJ2AMODJZHASE5A3OTMX4WOEYSGOSMV3MPWQPUGOGTA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: <nil> Sep 13 14:02:16.744335828 (168h33m52.783908959s) [seg_pos={0 0}, upload=148215]: Segment reached optimal threshold of 13 pieces. Sep 13 14:02:16.744608350 (168h33m52.78418147s) [noise=true , address=145.79.227.6:4259 , piece_id=PKWTWHUWLQUNAZSV3JYHKXPO3YJ4BQ7BJHFSVZJGDYLHISEDW6SA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->145.79.227.6:4259: use of closed network connection Sep 13 14:02:16.744656332 (168h33m52.784229452s) [noise=true , address=45.91.116.5:4297 , piece_id=ICCUEV23V6TAUVC23CKHY7HNPARLG7ZLYCOSDW5TELGHNHGWZI2Q, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50216->45.91.116.5:4297: use of closed network connection Sep 13 14:02:16.744677272 (168h33m52.784250392s) [noise=true , address=145.79.227.14:4840 , piece_id=V3M2XHU4LI5GHD3EOBSL7VGYW2TTLXUBBR2ARFZUPP3I62HHZGLA, seg_pos={0 0}, upload=148215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: expected piece hash; context canceled; EOF Sep 13 14:02:16.744731005 (168h33m52.784304125s) [seg_pos={0 0}, upload=148215]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 13 14:02:16.744857256 (168h33m52.784430386s) [upload=148215]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 13 14:02:16.744866254 (168h33m52.784439374s) [upload=148215]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 13 14:02:27.229168566 (168h34m3.268741686s) [upload=148215]: Done uploading segments... Sep 13 14:02:27.229191870 (168h34m3.268765s) [upload=148215]: Upload finished. err: <nil> Duration: 6.436256816s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........................X......X...........X......X.XX.XX.X] duration: 30.001199682s FAIL (2025-09-13 14:13:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001221313s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)