Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.758576624s FAIL (2025-09-23 16:18:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.758623715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.180653139s FAIL (2025-09-23 16:18:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.180686142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.947950446s FAIL (2025-09-23 16:17:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.947990231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.039650999s FAIL (2025-09-23 16:16:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.039707025s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.09967635s FAIL (2025-09-23 16:15:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.099709533s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.353739447s FAIL (2025-09-23 16:14:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.353784573s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.277727837s FAIL (2025-09-23 16:13:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.277761682s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.084270022s FAIL (2025-09-23 16:12:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.084302113s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.13104663s FAIL (2025-09-23 16:12:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.131077208s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.077712032s FAIL (2025-09-23 16:11:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.077744624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.021144308s FAIL (2025-09-23 16:10:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.021178364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.149060593s FAIL (2025-09-23 16:09:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.149180803s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.357678966s FAIL (2025-09-23 16:09:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 7.357710386s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002727552s FAIL (2025-09-23 16:09:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 30.002750665s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 315.654893ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:34:07 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 315.683079ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 400.890329ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:32:52 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 400.942177ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 263.70855ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:31:39 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 263.778813ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 399.21553ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:30:48 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 399.2472ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 370.146137ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:30:03 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 370.218194ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 258.437993ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:29:01 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 258.472159ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 419.237266ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:28:03 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 419.265479ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 259.422097ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:27:07 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 259.453547ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 269.133801ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:26:42 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 269.163738ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 267.156005ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:26:23 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 267.199486ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 270.728707ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:25:59 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 270.758474ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 253.500098ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:25:20 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 253.527391ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 270.039798ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:24:51 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 270.070146ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 309.029048ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:23:58 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 309.055117ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 304.475196ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:23:06 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 304.627306ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 264.907623ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:21:55 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 264.962217ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 261.595045ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:21:14 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 261.627677ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 267.473689ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:20:00 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 267.506801ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 274.431067ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:19:40 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 274.497584ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 281.445032ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:19:02 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 281.50172ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 441.643734ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:17:57 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 441.674482ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 369.557023ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:16:43 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 369.621244ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 258.491996ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:15:52 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 258.534427ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 275.743808ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:15:11 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 275.775236ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 275.133896ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:14:28 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 275.163041ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 342.351468ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:13:42 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 342.381827ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 316.420631ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:12:47 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 316.453334ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 9.39065924s FAIL (2025-09-23 15:12:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 14:30:47.519377614 (59m32.981470097s) [upload=879]: Starting upload Sep 23 14:30:47.519450904 (59m32.981543377s) [upload=879]: Done starting upload Sep 23 14:30:47.519622370 (59m32.981714853s) [upload=879]: Uploading segments... Sep 23 14:30:47.519856386 (59m32.981948869s) [upload=879]: Got next segment. Inline: false Sep 23 14:30:47.519972166 (59m32.982064649s) [upload=879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 14:30:47.519982527 (59m32.982075s) [upload=879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 14:30:47.907384470 (59m33.369476953s) [seg_pos={0 0}, upload=879]: Begin upload segment... Sep 23 14:30:47.907442832 (59m33.369535305s) [seg_pos={0 0}, upload=879]: Done begin upload segment. Sep 23 14:30:47.907468551 (59m33.369561034s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.907501273 (59m33.369593746s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908302589 (59m33.370395092s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908905226 (59m33.370997709s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908915045 (59m33.371007528s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909511421 (59m33.371603894s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909628834 (59m33.371721317s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910120030 (59m33.372212513s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910592179 (59m33.372684662s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911055531 (59m33.373148014s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911418722 (59m33.373511205s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911765573 (59m33.373858046s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911831599 (59m33.373924072s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912323926 (59m33.374416409s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912999052 (59m33.375091535s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.913617148 (59m33.375709632s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:50.224478053 (59m35.686570536s) [upload=879]: Next returned nil segment Sep 23 14:30:50.224519211 (59m35.686611684s) [upload=879]: Waiting for error group managing segments... Sep 23 14:30:51.114370090 (59m36.576462563s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.115043943 (59m36.577136426s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.158463131 (59m36.620555614s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.312220734 (59m36.774313217s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.564121960 (59m37.026214443s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.618986272 (59m37.081078755s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:52.283737671 (59m37.745830144s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:56.761671991 (59m42.223764474s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:02.511456285 (59m47.973548768s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:04.580071107 (59m50.04216359s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:07.593753483 (59m53.055845956s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:16.248941040 (1h0m1.711033523s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection Sep 23 14:31:16.249042443 (1h0m1.711134916s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249119410 (1h0m1.711211883s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection Sep 23 14:31:16.249206846 (1h0m1.711299319s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection Sep 23 14:31:16.249258325 (1h0m1.711350808s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249284624 (1h0m1.711377107s) [seg_pos={0 0}, upload=879]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 14:31:16.637266714 (1h0m2.099359197s) [upload=879]: Done uploading segments... Sep 23 14:31:16.637306770 (1h0m2.099399253s) [upload=879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 9.390683396s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 29.389825066s FAIL (2025-09-23 15:12:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 14:30:47.519377614 (59m32.981470097s) [upload=879]: Starting upload Sep 23 14:30:47.519450904 (59m32.981543377s) [upload=879]: Done starting upload Sep 23 14:30:47.519622370 (59m32.981714853s) [upload=879]: Uploading segments... Sep 23 14:30:47.519856386 (59m32.981948869s) [upload=879]: Got next segment. Inline: false Sep 23 14:30:47.519972166 (59m32.982064649s) [upload=879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 14:30:47.519982527 (59m32.982075s) [upload=879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 14:30:47.907384470 (59m33.369476953s) [seg_pos={0 0}, upload=879]: Begin upload segment... Sep 23 14:30:47.907442832 (59m33.369535305s) [seg_pos={0 0}, upload=879]: Done begin upload segment. Sep 23 14:30:47.907468551 (59m33.369561034s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.907501273 (59m33.369593746s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908302589 (59m33.370395092s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908905226 (59m33.370997709s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908915045 (59m33.371007528s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909511421 (59m33.371603894s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909628834 (59m33.371721317s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910120030 (59m33.372212513s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910592179 (59m33.372684662s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911055531 (59m33.373148014s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911418722 (59m33.373511205s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911765573 (59m33.373858046s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911831599 (59m33.373924072s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912323926 (59m33.374416409s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912999052 (59m33.375091535s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.913617148 (59m33.375709632s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:50.224478053 (59m35.686570536s) [upload=879]: Next returned nil segment Sep 23 14:30:50.224519211 (59m35.686611684s) [upload=879]: Waiting for error group managing segments... Sep 23 14:30:51.114370090 (59m36.576462563s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.115043943 (59m36.577136426s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.158463131 (59m36.620555614s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.312220734 (59m36.774313217s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.564121960 (59m37.026214443s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.618986272 (59m37.081078755s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:52.283737671 (59m37.745830144s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:56.761671991 (59m42.223764474s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:02.511456285 (59m47.973548768s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:04.580071107 (59m50.04216359s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:07.593753483 (59m53.055845956s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:16.248941040 (1h0m1.711033523s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection Sep 23 14:31:16.249042443 (1h0m1.711134916s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249119410 (1h0m1.711211883s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection Sep 23 14:31:16.249206846 (1h0m1.711299319s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection Sep 23 14:31:16.249258325 (1h0m1.711350808s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249284624 (1h0m1.711377107s) [seg_pos={0 0}, upload=879]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 14:31:16.637266714 (1h0m2.099359197s) [upload=879]: Done uploading segments... Sep 23 14:31:16.637306770 (1h0m2.099399253s) [upload=879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 29.389860685s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 8.208369204s FAIL (2025-09-23 15:11:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 14:30:47.519377614 (59m32.981470097s) [upload=879]: Starting upload Sep 23 14:30:47.519450904 (59m32.981543377s) [upload=879]: Done starting upload Sep 23 14:30:47.519622370 (59m32.981714853s) [upload=879]: Uploading segments... Sep 23 14:30:47.519856386 (59m32.981948869s) [upload=879]: Got next segment. Inline: false Sep 23 14:30:47.519972166 (59m32.982064649s) [upload=879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 14:30:47.519982527 (59m32.982075s) [upload=879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 14:30:47.907384470 (59m33.369476953s) [seg_pos={0 0}, upload=879]: Begin upload segment... Sep 23 14:30:47.907442832 (59m33.369535305s) [seg_pos={0 0}, upload=879]: Done begin upload segment. Sep 23 14:30:47.907468551 (59m33.369561034s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.907501273 (59m33.369593746s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908302589 (59m33.370395092s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908905226 (59m33.370997709s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908915045 (59m33.371007528s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909511421 (59m33.371603894s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909628834 (59m33.371721317s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910120030 (59m33.372212513s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910592179 (59m33.372684662s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911055531 (59m33.373148014s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911418722 (59m33.373511205s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911765573 (59m33.373858046s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911831599 (59m33.373924072s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912323926 (59m33.374416409s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912999052 (59m33.375091535s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.913617148 (59m33.375709632s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:50.224478053 (59m35.686570536s) [upload=879]: Next returned nil segment Sep 23 14:30:50.224519211 (59m35.686611684s) [upload=879]: Waiting for error group managing segments... Sep 23 14:30:51.114370090 (59m36.576462563s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.115043943 (59m36.577136426s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.158463131 (59m36.620555614s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.312220734 (59m36.774313217s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.564121960 (59m37.026214443s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.618986272 (59m37.081078755s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:52.283737671 (59m37.745830144s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:56.761671991 (59m42.223764474s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:02.511456285 (59m47.973548768s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:04.580071107 (59m50.04216359s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:07.593753483 (59m53.055845956s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:16.248941040 (1h0m1.711033523s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection Sep 23 14:31:16.249042443 (1h0m1.711134916s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249119410 (1h0m1.711211883s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection Sep 23 14:31:16.249206846 (1h0m1.711299319s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection Sep 23 14:31:16.249258325 (1h0m1.711350808s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249284624 (1h0m1.711377107s) [seg_pos={0 0}, upload=879]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 14:31:16.637266714 (1h0m2.099359197s) [upload=879]: Done uploading segments... Sep 23 14:31:16.637306770 (1h0m2.099399253s) [upload=879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 8.20839881s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 263.397027ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:11:37 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 263.423767ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 9.550555676s FAIL (2025-09-23 15:11:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 14:30:47.519377614 (59m32.981470097s) [upload=879]: Starting upload Sep 23 14:30:47.519450904 (59m32.981543377s) [upload=879]: Done starting upload Sep 23 14:30:47.519622370 (59m32.981714853s) [upload=879]: Uploading segments... Sep 23 14:30:47.519856386 (59m32.981948869s) [upload=879]: Got next segment. Inline: false Sep 23 14:30:47.519972166 (59m32.982064649s) [upload=879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 14:30:47.519982527 (59m32.982075s) [upload=879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 14:30:47.907384470 (59m33.369476953s) [seg_pos={0 0}, upload=879]: Begin upload segment... Sep 23 14:30:47.907442832 (59m33.369535305s) [seg_pos={0 0}, upload=879]: Done begin upload segment. Sep 23 14:30:47.907468551 (59m33.369561034s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.907501273 (59m33.369593746s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908302589 (59m33.370395092s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908905226 (59m33.370997709s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908915045 (59m33.371007528s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909511421 (59m33.371603894s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909628834 (59m33.371721317s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910120030 (59m33.372212513s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910592179 (59m33.372684662s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911055531 (59m33.373148014s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911418722 (59m33.373511205s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911765573 (59m33.373858046s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911831599 (59m33.373924072s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912323926 (59m33.374416409s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912999052 (59m33.375091535s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.913617148 (59m33.375709632s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:50.224478053 (59m35.686570536s) [upload=879]: Next returned nil segment Sep 23 14:30:50.224519211 (59m35.686611684s) [upload=879]: Waiting for error group managing segments... Sep 23 14:30:51.114370090 (59m36.576462563s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.115043943 (59m36.577136426s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.158463131 (59m36.620555614s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.312220734 (59m36.774313217s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.564121960 (59m37.026214443s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.618986272 (59m37.081078755s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:52.283737671 (59m37.745830144s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:56.761671991 (59m42.223764474s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:02.511456285 (59m47.973548768s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:04.580071107 (59m50.04216359s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:07.593753483 (59m53.055845956s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:16.248941040 (1h0m1.711033523s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection Sep 23 14:31:16.249042443 (1h0m1.711134916s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249119410 (1h0m1.711211883s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection Sep 23 14:31:16.249206846 (1h0m1.711299319s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection Sep 23 14:31:16.249258325 (1h0m1.711350808s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249284624 (1h0m1.711377107s) [seg_pos={0 0}, upload=879]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 14:31:16.637266714 (1h0m2.099359197s) [upload=879]: Done uploading segments... Sep 23 14:31:16.637306770 (1h0m2.099399253s) [upload=879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 9.550589159s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 257.335434ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:11:19 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 257.378276ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 273.171406ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:10:48 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 273.19943ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.956600215s FAIL (2025-09-23 15:10:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 14:30:47.519377614 (59m32.981470097s) [upload=879]: Starting upload Sep 23 14:30:47.519450904 (59m32.981543377s) [upload=879]: Done starting upload Sep 23 14:30:47.519622370 (59m32.981714853s) [upload=879]: Uploading segments... Sep 23 14:30:47.519856386 (59m32.981948869s) [upload=879]: Got next segment. Inline: false Sep 23 14:30:47.519972166 (59m32.982064649s) [upload=879]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 14:30:47.519982527 (59m32.982075s) [upload=879]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 14:30:47.907384470 (59m33.369476953s) [seg_pos={0 0}, upload=879]: Begin upload segment... Sep 23 14:30:47.907442832 (59m33.369535305s) [seg_pos={0 0}, upload=879]: Done begin upload segment. Sep 23 14:30:47.907468551 (59m33.369561034s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.907501273 (59m33.369593746s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908302589 (59m33.370395092s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908905226 (59m33.370997709s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.908915045 (59m33.371007528s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909511421 (59m33.371603894s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.909628834 (59m33.371721317s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910120030 (59m33.372212513s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.910592179 (59m33.372684662s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911055531 (59m33.373148014s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911418722 (59m33.373511205s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911765573 (59m33.373858046s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.911831599 (59m33.373924072s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912323926 (59m33.374416409s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.912999052 (59m33.375091535s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:47.913617148 (59m33.375709632s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Uploading piece... Sep 23 14:30:50.224478053 (59m35.686570536s) [upload=879]: Next returned nil segment Sep 23 14:30:50.224519211 (59m35.686611684s) [upload=879]: Waiting for error group managing segments... Sep 23 14:30:51.114370090 (59m36.576462563s) [noise=true , address=45.91.116.9:4261 , piece_id=PQPCE2UWB6ZKAUM5HLJT4LD4KVVR5A2BGMRSAKQ3SC3B4LXIURCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.115043943 (59m36.577136426s) [noise=true , address=45.91.116.10:4082 , piece_id=M2UNRTPS3MA3GFUDWT5LZCAVU2OEBRTU4LDK4JLICR65JI5PM44A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.158463131 (59m36.620555614s) [noise=true , address=77.90.31.11:4553 , piece_id=LX72FGFAYXQYJPNT5HE4CV3UXES7TIYNGHPTB34U3UW653SNENLA, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.312220734 (59m36.774313217s) [noise=true , address=45.91.116.5:4298 , piece_id=DLAQYV5PNI2MSRRO7XVMMIJ32GUKUZ7TDE5Q3CB5GES2RN7JHCPQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.564121960 (59m37.026214443s) [noise=true , address=45.91.116.11:4119 , piece_id=W3RW4VIQL3NGHZWANJFPATJB5BQGRR7ZS2ORJBPXVRXSZO72JWBQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:51.618986272 (59m37.081078755s) [noise=true , address=45.91.116.13:4153 , piece_id=2ROXASZOUNQWODFA5LSORBVRL3QKK7TVLMJ2GXAVSB5VQ3QSS5MQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:52.283737671 (59m37.745830144s) [noise=true , address=45.91.116.6:4975 , piece_id=FHB2P3FIO7B5ZZS525I2LQXXB4KDOPXA5XYNWMZHQDIUMVXZX6WQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:30:56.761671991 (59m42.223764474s) [noise=true , address=145.79.227.12:4767 , piece_id=VLAIEHPFPVH2NA26HYKORPY4AOECYK5NSNEPR6TZFH5FO37SHG5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:02.511456285 (59m47.973548768s) [noise=true , address=145.79.227.4:4185 , piece_id=WZOOXZYG7OP5B3T7CKUCDHGEQQOH35COQAT6EGL4PLNETXTLGMZQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:04.580071107 (59m50.04216359s) [noise=true , address=145.79.227.17:4969 , piece_id=C7HRWDYQ22JPV2W53E56F6AL47TEUJE5DYEY6IFJWUN54ETFTS3Q, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:07.593753483 (59m53.055845956s) [noise=true , address=145.79.227.19:4079 , piece_id=H2ZXGOQREBCAFDXV4FW6GQMOKBRRMKQDVGYCVDTDF5M3RNL6D4JQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: <nil> Sep 23 14:31:16.248941040 (1h0m1.711033523s) [noise=true , address=145.79.227.7:4294 , piece_id=IRRMTZS22642CHFXWNRJCUYVQKN2ELFNNKDABCLWRJGJVBSIRXGQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60786->145.79.227.7:4294: use of closed network connection Sep 23 14:31:16.249042443 (1h0m1.711134916s) [noise=true , address=145.79.227.16:4909 , piece_id=XESCG3FTTZE4HAWGAKIUC6XTVYUZFTBYFYAE6ZZ7ZBVPBFGHSC5A, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249119410 (1h0m1.711211883s) [noise=true , address=145.79.227.15:4874 , piece_id=EPRTDN3BZ7ECEED22SAAOX3YWCMEWGACGY33KREZWOV6Z6QDKLCQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33828->145.79.227.15:4874: use of closed network connection Sep 23 14:31:16.249206846 (1h0m1.711299319s) [noise=true , address=145.79.227.6:4259 , piece_id=COFBXVCWOAZQYOVPLTERLC422JN6YKD6OO3S7HZ5OTIQTDAC6YYQ, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54632->145.79.227.6:4259: use of closed network connection Sep 23 14:31:16.249258325 (1h0m1.711350808s) [noise=true , address=145.79.227.5:4223 , piece_id=TTDEHY2UNUCWCZYGAKXRR23PI2YO5M3NOTERFKFGIINPYFI356WA, seg_pos={0 0}, upload=879]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 14:31:16.249284624 (1h0m1.711377107s) [seg_pos={0 0}, upload=879]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 14:31:16.637266714 (1h0m2.099359197s) [upload=879]: Done uploading segments... Sep 23 14:31:16.637306770 (1h0m2.099399253s) [upload=879]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.9566338s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 263.540879ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:09:45 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 263.568312ms edge gateway vhost bgp us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)